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FOREWORD 

Positive Behavioral Support and Applied Behavior Analysis 

Todd R. Risley, PhD 

University of Alaska 



The dimensions of Applied Behavior Analysis were developed by Montrose 
Wolf and his colleagues in a remarkable series of exploratory studies, across a 
variety of children and problems, at the University of Washington in 1962-64. 
Those studies modeled how to arrange interventions in clinics, playgrounds, 
classrooms, and homes. They demonstrated the speed and power to change 
behavior of procedures based on the principles of behavior as delineated by 
B. F. Skinner. And they developed measurement tactics and experimental 
design variations to fit each new real-world condition and problem behavior 
encountered (see Risley, 1997 for a listing of these studies). It is important to 
note the three unprecedented and audacious features of those pioneering 
studies: 1) deliberate interventions in the daily lives of people; 2) fast and large 
behavior change; and 3) scientific documentation of field research. With the 
institution of the Journal of Applied Behavior Analysis in 1968, each of these 
features has been narrowed and codified across succeeding generations of 
researchers. 

Thirty years later this monograph — “Positive Behavior Support for People 
With Developmental Disabilities: A Research Synthesis”— has reviewed the last 
10 years of a major branch of Applied Behavior Analysis and has recommended 
some changes in direction. Their review is carefully designed and conducted— 
this is not biased assertion, but an objective meta-analysis with well defined, 
reliable variables. (Its clarity and sophistication are such that I regard it as a 
model of logic and methodology for Graduate research courses.) Their 
findings are presented with caution— this is not polemic or salesmanship, but 
reasoned consideration. (In fact, I think it understates the evidence favoring 
Positive Behavioral Support strategies.) Their recommendations are wise and 
practical — this is not pie-in-the-sky wishful thinking but a useful guide to the 
next generation of research, service, and policy. (I am gratified by this evidence 
of wisdom, leadership, and cooperation in the third and fourth generations of 
applied behavioral researchers.) 

In their recommendations they state that we need a new applied science. 
Thirty-five years ago Montrose Wolf and a few others said the same— and 
proceeded to found Applied Behavior Analysis with only a little experience 
from which to deduce its characteristics. In contrast, these 10 authors repre- 
sent scores of other researchers and have examined hundreds of studies. Their 
basis for recommending the dimensions of future Positive Behavioral Support 
is simply a conservative extrapolation of information. 

ERIC 
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Some of their recommendations to researchers suggest only a more 
systematic approach to things currently done. “Consumer needs” and “Partici- 
patory Action Research is an (admittedly large) extension in the effort and 
timing of Social Validity” If functional assessment is to have ecological 
validity it must be done throughout life, and therefore, “hypotheses” of antece- 
dent and consequence functions can often be tested by close monitoring during 
interventions. Goals of life improvement and long-term benefits are now 
frequently (if poorly) addressed in anecdotal reports supplementing primary 

However, there are fundamental attributes of Applied Behavior Analysis 
with which their recommendations conflict: procedural specificity, and 
demonstration of causality. Are these attributes of Applied Behavior Analysis 
fundamental to the advancement of knowledge? Perhaps not as much as we 
assume. 

The recommendations for multicomponent interventions, individually 
adapted to circumstances and revised over time, is contrary to the goals of 
specifiable treatment packages” or “manualized” treatments or “model” 
treatment programs common in Applied Behavior Analysis. Some specificity of 
intervention is, of course, desirable but to what use has it been put in 30 years 
of behavioral research? 

This monograph, itself, is a most rare example of careful comparison of 
procedures across studies. More commonly we have quite similar procedures 
with different proprietary labels, and common labels masking large differ- 
ences between intervention programs. Other areas of science and technology 
that have different customs for analyzing procedures across researchers should 
be examined for guidance; as should field trial protocols which specify 
treatments by their adherence to a set of decision rules rather than a set of 
fixed procedures. 

The recommendations for larger interventions, goals, and measures cannot 
be met while adhering to high requirements of experimental manipulation. 
Despite occasional articles demonstrating or advocating the usefulness of 
group designs, correlations, naturalistic observations, and case studies in the 
Journal of Applied Behavior Analysis, demonstrations of causality over time 
have remained the standard for participation in Applied Behavior Analysis. It is 
this doctrine that most biases applied behavior analysis research toward more 
manageable contexts and problems. The demonstration of causality between 
independent and dependent variables through repeated manipulation of 
independent variables across time defined the Experimental Analysis of 
Behavior. The founders of Applied Behavior Analysis were active participants 
in the Experimental Analysis of Behavior journal and meetings. They never 
considered any other research logic. Some of the founders went so far as to 
define Applied Behavior Analysis as the use of single-subject time-series 
designs. Others disagreed and considered any research design to be simply a 
tool, to be chosen and used when needed. 



It is clear that Positive Behavioral Support researchers need to use other 
tools in addition to experimental manipulations. Will Applied Behavior 
Analysis accommodate this, or will there become two distinct groups one, 
allied to the Experimental Analysis of Behavior, and the other, dedicated to 
empirical problem-solving? It should be noted that empirical problem solving 
was the original impetus for the studies on which Applied Behavior Analysis 
was founded; problems were addressed as they presented themselves, measure- 
ment and research design considerations were overlaid to achieve the best 
information the circumstances allowed; the resulting publications were 
carefully considered reports of the problem-solving process, with due attention 
to threats to internal and external validity — not experiments proving a point. 

A healthy dose of Donald Campbell’s consideration of problem-solving 
research (1957) and reforms as experiments (1969) would help the enterprise 
of Positive Behavioral Support pursue the recommendations proposed in this 
monograph and to return to the roots of Applied Behavior Analysis. 



Campbell, D.T. (1957). Factors relevant to the validity of experiments in social settings. 
Psychological Bulletin , 54 , 297-312. 

Campbell, D. T. (1969). Reforms as experiments. American Psychologist, 24, 409-29. 
Risley.T. R. (1997). Montrose M.Wolf: The origin of the dimensions of Applied Behavior 
Analysis, Journal of Applied Behavior Analysis, 30, 377-381. 
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FOREWORD 

Madeleine Will, Former Assistant Secretary, OSERS 

U.S . Department of Education 



There are occasions when science confirms aspirations, when research justifies 
conviction. The analysis by Carr et al. is one of those. 

It is a matter of record that during the time when I served as Assistant 
Secretary of Education, Office of Special Education and Rehabilitative Services 
( 1983 - 1989), professionals, parents of children and adults with disabilities, self- 
advocates, and policy makers were locked in fierce debate concerning four 
matters on which I had chosen to take a leadership role: (a) inclusion of 
students with disabilities in regular education, (b) transition of students with 
disabilities from high school to work, (c) withholding or withdrawal of 
efficacious medical treatment from newborns who had obvious disabilities, 
and (d) the use of aversive behavioral interventions with children and adults 
with disabilities. In each of these issues policy decisions were made based on 
both technology and hope, on science and aspiration. In each case there was a 
capacity to do the right thing. More than that, there was a hope shared by 
myself and many others, that policy eventually would dignify the lives of people 
with disabilities by granting them new rights. This was especially true in the 
area of behavior support. 

As I read the present monograph on Positive Behavior Support I could not 
help but recall the focus of our policy struggle in the late 1980s. Our efforts to 
match our aspirations with existing science were hindered, as is often the case, 
by the absence of science about the use of positive behavioral interventions. 

The monograph you hold before you is in many ways the document we wanted 
a decade ago. Here is a careful analysis documenting that positive behavioral 
procedures can produce important change in the behavior and lives of people 
with disabilities. Positive interventions can be effective. 

Simply put: the monograph objectively reviews the published research on 
positive behavioral interventions and draws conclusions from this database. 

The organization and presentation of the published research is of special 
importance because it allows each reader to consider the conclusions in light of 
their personal interpretation of the data. 

What I myself take from this reyiew is that positive behavioral interven- 
tions are indeed beneficial; that they complement other practices that are now 
codified into policy (inclusion, transition to real work, and access to medical 
treatment); and that positive interventions are justified now on scientific 
grounds, just as some of us thought, long ago and still, that they are com- 
manded on humane, moral, and constitutional grounds. Here, at last, is the 



proof positive of their efficacy. My son, Jonathan Will, and many other citizens 
with disabilities now have available to them (should they need it) a promising 
science that it was my privilege to stimulate and support. In a very practical 
way this monograph defines the science that confirms our aspiration; the 
research that justifies our convictions. 
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Overview 



OVERVIEW 



This monograph was prepared in 
response to a request from the United 
States Department of Education, Office of 
Special Education Programs, for a review 
of the literature on positive behavior 
support that provided (a) a definition of 
the approach, (b) an analysis of the 
database, (c) a delineation of gaps in our 
knowledge, and (d) suggestions for future 
directions. Because of the scope and 
complexity of the positive behavior 
support approach, it will be useful to 
begin with an overview of the main 
content areas of our monograph. 

Positive behavior support (PBS) is an 
approach for dealing with problem 
behavior that focuses on the remediation 
of deficient contexts (i.e., environmental 
conditions and/or behavioral repertoires) 
that by functional assessment are 
documented to be the source of the 
problem. The research published on PBS 
between 1985 and 1996 was reviewed 
with respect to four categories of vari- 
ables: demographics, assessment 
practices, intervention strategies, and 
outcomes. 

The data derived from examining 
these four categories of variables were 



analyzed to answer five questions: 

(a) How widely applicable is PBS? (b) In 
what ways is the field evolving? (c) How 
effective is PBS? (d) What factors 
modulate the effectiveness of PBS? 

(e) How responsive is the PBS literature to 
the needs of consumers? 

Results indicated that (a) PBS is 
widely applicable to people with serious 
problem behavior; (b) the field is growing 
rapidly overall, but especially in the use of 
assessment and in interventions that 
focus on correcting environmental 
deficiencies; (c) using stringent criteria of 
success, PBS is effective in reducing 
problem behavior in one half to two 
thirds of the cases; (d) success rates 
nearly double when intervention is based 
on a prior functional assessment; and 
(e) consumer needs that emphasize 
comprehensive lifestyle support, long- 
term change, practicality and relevance, 
and direct support for the consumers 
themselves are inadequately addressed by 
the research base. 

Recommendations are made for 
bridging the research-to-practice gap in 
the areas of research priorities, service 
provision, social policy, and governmental 
action. 
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Introduction 



CHAPTER 1 

INTRODUCTION 



A new era of behavioral support is 
emerging. It is building from the careful, 
rigorous science of the past three decades 
and the practical demands of families, 
teachers, and clinicians. The new 
approach, called positive behavioral 
support (Koegel, Koegel, & Dunlap, 1996), 
is evolving rapidly as new information 
and challenges arise. The volume of 
recent empirical work suggests the need 
for a synthesis of the current research 
literature. The purpose of this research 
synthesis is to (a) define positive behav- 
ioral support, (b) provide a systematic 
analysis of the existing research database 
with respect to positive behavioral 
support, (c) compare that database with 
current clinical needs in the field, and 
(d) suggest future directions for research, 
practice, and policy. Our focus is on the 
application of behavioral support for 
people with developmental disabilities 
and autism. 

We provide an introduction to the 
research synthesis by presenting (a) a 
brief overview and definition of positive 
behavior support, (b) the rationale for the 
present research synthesis of positive 
behavior support, (c) the research 
questions posed, and (d) the structure of 
the research synthesis. 

Positive Behavioral 
Support: Overview and 
Definition 

Problem behaviors such as aggression, 
self-injury, tantrums, and property 
destruction have long been barriers to 
successful education, socialization, 



employment, and community adaptation 
(Meyer, Peck, & Brown, 1991; 
Scheerenberger, 1990; White, Lakin, 
Bruininks, &Li, 1991). The goal of 
positive behavior support (PBS) is to 
apply behavioral principles in the 
community in order to reduce problem 
behaviors and build appropriate behav- 
iors that result in durable change and a 
rich lifestyle. The foundation of PBS lies 
in early efforts to apply principles of 
behavior to improve the lives of children 
with severe problem behaviors (Bijou 8c 
Baer, 1961 ; Bijou, Peterson, 8c Ault, 1968; 
Browning 8c Stover, 1971 ). The lawful 
relationships between behavior and 
environment were applied to people with 
real problems. The results were encourag- 
ing and led to the development in the 
1960s and 1970s of an array of interven- 
tion procedures (Barrett, 1986; Foxx, 

1982; Kazdin, 1980). Each of these 
procedures proved successful at reducing 
problem behaviors in some situations and 
unsuccessful in others. 

The need to improve the application 
of intervention procedures led to a renewed 
appreciation for the earlier call to organize 
interventions based on a careful functional 
assessment of the problem behavior 
(Baer, Wolf, 8c Risley, 1968; Bijou et al., 
1968). Beginning in the late 1970s (Carr, 
1977) and continuing through today, the 
concept of functional assessment has 
been transformed into a practical 
technology for guiding the development 
of behavioral interventions (Bailey 8c 
Pyles, 1989; Gardner 8c Sovner, 1994; 
O’Neill, Horner, Albin, Storey, 8c Sprague, 
1997a 8c b; Reichle 8c Wacker, 1993). 
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A central result has been an expan- 
sion of interventions beyond those based 
on consequence manipulations to include 

(a) altering the environment before 
problem behaviors occur, and 

(b) teaching appropriate behaviors as an 
effective strategy for reducing unwanted 
behaviors. Behavioral support is becom- 
ing less a process of selecting an interven- 
tion, and more the construction of a 
comprehensive set of procedures that 
include change of the environment to 
make problem behaviors irrelevant, 
instruction on appropriate behaviors that 
makes the problem behavior inefficient, 
and manipulation of consequences to 
ensure that appropriate behaviors are 
more consistently and powerfully 
reinforced than are problem behaviors. As 
the structure of behavioral support 
expands, so does recognition that a 
complete technology will also require 
attention to those interacting variables in 
a setting/system that affect the imple- 
mentation of effective procedures (Sailor, 
1996; Taylor-Greene et al., 1997). 

As effective approaches to behavioral 
support emerged, expectations for the 
outcomes and acceptability of the 
technology were redefined. The initial 
focus of behavioral interventions was on 
simple reduction of problem behaviors. 

As this proved possible, yet insufficient, 
expectations changed. Through reexami- 
nation of original assumptions (Baer et 
al., 1968) and attention to the messages 
provided by real-world users of the 
technology, investigators expanded 
functional assessment technology and the 
expectations for behavioral interventions 
were redefined. Effective behavioral 
support needed not only to reduce 
problem behaviors, but also to build 
prosocial behavior, document durable 
change, generalize across the full range of 

O 



situations an individual encountered, and 
produce access to a rich lifestyle. 

This is the key concept that defines 
PBS: To remediate problem behavior, it is 
necessary first to remediate deficient 
contexts. There are two kinds of deficien- 
cies: those relating to environmental 
conditions, and those relating to behavior 
repertoires. Environmental conditions are 
deficient to the extent that they involve 
lack of choice, inadequate teaching 
strategies, minimal access to engaging 
materials and activities, poorly selected 
daily routines, and a host of other 
proximal and distal antecedent stimuli 
related to the previous factors. Behavior 
repertoires are deficient to the extent that 
communication skills, self-management, 
social skills, and other constructive 
behaviors are inadequately developed or 
absent. (The constituent elements relating 
to the two kinds of deficiencies just 
summarized are defined at length in 
Chapter 2, under the subhead Interven- 
tion Strategies.) Recently there has been 
much discussion concerning the strong 
association between these two types of 
deficiencies and the display of problem 
behavior; many have concluded that 
problem behavior can be effectively 
addressed by focusing on the assessment 
and remediation of context (Emerson, 
McGill, & Mansell, 1994; Koegel et al., 

1996; Luiselli & Cameron, 1998; Lutzker 
8c Campbell, 1994; Reichle 8c Wacker, 

1993). 

As noted, the primary focus of the 
field has not always been on context per 
se. For many years, researchers and 
clinicians alike emphasized strategies that 
concentrated on the problem behavior 
itself (e.g., aggression, self-injury, and 
property destruction), rather than on 
deficiencies such as poor environmental 
conditions or a lack of functional skills. 
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This emphasis led to the development of a 
wide variety of reactive (often punitive) 
interventions designed to suppress or 
eliminate the problem behavior directly 
(Bucher & Lovaas, 1968; Cataldo, 1991; 
Harris & Ersner-Hershfield, 1978; Matson 
& DiLorenzo, 1984). The relative merits of 
reactive versus contextually based 
interventions have been hotly debated in 
the literature (Guess, Helmstetter, 

Turnbull, & Knowlton, 1987; Repp & 

Singh, 1990). Yet people on all sides of this 
debate have always acknowledged that the 
ultimate goal is not simply the reduction 
of problem behavior, but rather improv- 
ing people’s lives. There is now wide- 
spread appreciation of the fact that the 
negative sequelae of problem behavior 
include not only physical danger to self 
and others, but also educational segrega- 
tion, limited employment opportunities, 
rejection by members of the community, 
separation from home and family, and, 
finally, social ostracism and a life without 
friendship (National Institutes of Health, 
1991). Clearly, behavioral technology 
must now become more comprehensive 
in scope and more cognizant of the role of 
the larger systems (e.g., family, school, 
employment, and funding) that influence 
the practicality of effective behavioral 
support (Sailor, 1996). Consideration of 
these larger issues has prompted interest 
in the kinds of environmental and 
behavioral context variables that are the 
essential characteristics of PBS. 

It may be worthwhile at this point to 
further distinguish PBS from other 
contemporary approaches. The two most 
substantive and frequently employed 
alternatives to PBS are the use of pharma- 
cotherapy (medication) and aversive 
procedures. As noted previously, the main 
focus of aversive procedures is the 
elimination, through punishment, of 




problem behavior. Such procedures are 
fundamentally and by definition reactive 
in nature. They are not employed until the 
problem behavior occurs. In contrast, PBS 
is proactive in nature. It is an attempt to 
remediate environmental and behavioral 
deficiencies so as to prevent future 
occurrences of the problem behavior. In 
sum, using aversive procedures conforms 
best to a crisis management paradigm; 
using PBS conforms best to a prevention 
paradigm. A detailed analysis of the 
aversives literature is ably presented 
elsewhere (Cataldo, 1991). 

The second non-PBS strategy, 
medication, involves the use of one or 
more drugs to suppress problem behavior. 
Often the drugs are administered over a 
long period of time, to address hypoth- 
esized or identified biochemical aberra- 
tions thought to underlie problem 
behavior. In this case, the use of medica- 
tion conforms to a preventive paradigm, 
because the successful use of medication 
would block the occurrence of future 
episodes of problem behavior. Sophisti- 
cated analyses of the voluminous and 
complex pharmacotherapy literature are 
also available elsewhere (Reiss 8c Aman, 
1998; Schaal & Hackenberg, 1994; 
Schroeder & Tessel, 1994; Thompson, 
Hackenberg, & Schaal, 1991). There is an 
obvious distinction between PBS and 
pharmacotherapy: Whereas the former 
focuses on the role of environmental 
factors in assessing and remediating 
problem behavior, the latter focuses on 
the role of biochemical factors. 

Some people question the advisabil- 
ity of comparing PBS with medication 
and the use of aversives. Our position, 
articulated in detail later, is that there are 
many good reviews of non-PBS strategies 
but a dearth of reviews concerning the 
unique contributions of PBS per se. 
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Therefore, we will focus our review on 
PBS alone. More important, making 
comparisons among these approaches 
would implicitly support what we believe 
is a false assumption: that one must 
always choose among the three ap- 
proaches because only one can be 
ascendant. Clinical experience would 
seem to contradict this assumption. For 
example, for many years, it has been 
considered a best practice to accompany 
the use of aversives with a detailed 
educational and social support plan that 
embodies the major features of PBS (Carr 
& Lovaas, 1983; Foxx, 1982, 1990; Foxx, 
Bittle, 8c Faw, 1989). Likewise, the 
literature on dual diagnosis indicates a 
potentially important role for medication. 
Specifically, some people with develop- 
mental disabilities may receive a secon- 
dary diagnosis such as depression, 
anxiety, bipolar disorder, or obsessive- 
compulsive disorder (Bodfish 8c Madison, 
1993; Lowry 8c Sovner, 1992; Ratey, 
Sovner, Parks, 8c Rogentine, 1991; Reiss 8c 
Rojahn, 1993). An emerging literature 
suggests that medication given to 
alleviate the symptoms associated with 
the secondary diagnosis may help reduce 
problem behavior (e.g., Bodfish 8c 
Madison, 1993; Sovner, 1989). As yet, 
there is no definitive research demon- 
strating a causal link between obsessive- 
compulsive disorder, for example, and 
problem behavior. Nonetheless, the 
empirical work on dual diagnosis sounds 
a cautionary note in that practitioners 
must consider the possibility that PBS 
alone might prove insufficient for dealing 
with individuals carrying diagnostic 
labels beyond developmental disabilities 
per se. Similarly, medication alone might 
prove insufficient for improving an 
individual’s lifestyle, one of the stated 
goals of PBS. 
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In sum, the three approaches for dealing 
with problem behavior do not exist 
independently of one another in 
practice. As yet, however, there is no 
well-developed research literature that 
explores, systematically, how the 
approaches interrelate. 

Need for a Review of 
Positive Behavior Support 

To date, there have been few synthesis 
reviews focused on PBS per se. This 
approach did not gain momentum until 
the mid-to-late 1980s, after which there 
was an explosive growth in the number of 
research studies, conceptual papers, and 
intervention manuals related to PBS (Carr 
et al., 1994; Donnellan, LaVigna, Negri- 
Shoultz, 8c Fassbender, 1988; Durand, 
1990; Evans 8c Meyer, 1985; Horner, 
Dunlap et al., 1990; LaVigna 8c Donnellan, 
1986; Meyer 8c Evans, 1989; Smith, 1990). 
There have been many excellent reviews 
dealing with the general issue of 
remediating problem behavior. Typically, 
however, reviewers combined the analysis 
of PBS with the analysis of other ap- 
proaches that differ from PBS, or they 
reviewed only a subset of PBS procedures 
(Didden, Duker, 8c Korzilius,1997; 
Lancioni 8c Hoogeveen, 1990; Lennox, 
Miltenberger, Spengler, 8c Erfanian, 1988; 
Matson 8c Taras, 1989; O’Brien 8c Repp, 
1990;Vollmer 8clwata, 1992). Conse- 
quently, previous reviews did not analyze, 
in depth, the unique contributions made 
by PBS per se. 

The dearth of synthesis reviews 
related to PBS was one reason for the 
National Institutes of Health commission- 
ing a Consensus Development Conference 
in 1989 to deal with the issue of destruc- 
tive behavior. One product of that 
conference was a synthesis review of PBS 
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based on the literature that existed 
through 1989 (Carr, Robinson, Taylor, 8c 
Carlson, 1990). Shortly thereafter a 
second analysis appeared reviewing the 
literature from 1976 to 1987 (Scotti, 

Evans, Meyer, & Walker, 1991). This 
analysis considered a number of issues 
relevant to PBS but devoted considerable 
coverage to non-PBS strategies as well. 
Finally, a recent review (Scotti, Ujcich, 
Weigle, Holland, & Kirk, 1996) analyzed 
intervention practices (but not outcomes) 
from 1988 to 1992 and extended the 
initial Scotti et al. (1991) review. Again, 
however, the analysis included non-PBS 
strategies. In sum, the absence, since 
1989, of a synthesis that focuses exclu- 
sively on PBS provided an important 
justification for undertaking the present 
review. 

Traditionally reviews have empha- 
sized issues of special interest to re- 
searchers: population demographics, 
experimental design considerations, 
assessment strategies, and measures of 
comparative intervention effectiveness. 

But as the field has matured, there has 
been greater appreciation for the perspec- 
tives of nonresearchers (consumers). In 
this review, we use the term consumers to 
include people with disabilities, their 
teachers, their friends, members of their 
families, administrators, and policy 
makers. The literature suggests that 
although PBS has made a major contribu- 
tion in dealing with the issue of problem 
behavior, a considerable gap exists 
between the needs and interests of 
researchers and nonresearcher consum- 
ers (Billingsley & Cross, 1991; Dunlap, 
Robbins, 8( Darrow, 1994; Haring, 1996; 
Horner, Diemer, & Brazeau, 1992; Sailor, 
1996; Turnbull 8( Turnbull, 1996). 
Considering this gap, we set out to 

determine how far the literature has come 

O 




in addressing this gap and, by implica- 
tion, what must be done next. This 
objective could be achieved only through 
a careful analysis of those parts of the 
PBS literature that, explicitly or implicitly, 
bear on the perspectives of 
nonresearchers. The relative absence of 
this type of analysis from previous 
reviews was another reason to examine 
the PBS literature as it has evolved to date. 

Research Questions Posed: 
Contributions of 
the Review 

In this section, we outline the major 
research questions posed in the review 
and the kinds of information derived 
from answering them. First, however, it is 
necessary to elaborate further on the 
defining characteristics of PBS so they 
can be systematically related to the key 
research questions. 

Elaboration of Defining 
Characteristics of PBS 

From the standpoint of the independent 
variable, the PBS approach refers to those 
interventions that involve altering 
deficient environmental conditions (e.g., 
activity patterns, choice options, prompt- 
ing procedures) and/or deficient behavior 
repertoires (e.g., communication, self- 
management, social skills). The alteration 
of environmental conditions can be 
achieved by modifying proximal stimuli 
(e.g., curriculum materials, prompts) or 
distal stimuli (e.g., rearranging the 
sequence of daily life routines). Because 
all such strategies focus on assessing and 
manipulating stimuli, we will refer to 
these strategies as stimulus-based 
interventions. The alteration of behavior 
repertoires can be achieved by modifying 
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socially appropriate, functional behaviors 
that are currently inadequately developed 
or absent (e.g., communication, job skills, 
social skills, independent living skills, 
self-management behavior). Because all 
strategies that increase the probability of 
such behaviors invariably involve a 
systematic and targeted application of 
reinforcement, we refer to these strategies 
as reinforcement-based intervention. The 
various parameters of stimulus-based 
and reinforcement-based interventions 
define the core of PBS with respect to the 
independent variables. 

The remediation of deficient contexts 
also helps to define the PBS approach 
with respect to dependent variables: 

(a) increased positive behavior, 

(b) improved lifestyle, and (c) decreased 
problem behavior. Specifically, as noted, 
stimulus- and reinforcement-based 
interventions are both designed to 
increase the probability that socially 
appropriate, functional behaviors (i.e., 
positive behaviors) will occur. For 
example, improvements made in instruc- 
tional procedures (a stimulus-based 
strategy) may increase correct academic 
responding (a positive behavior); that is, 
the revised instructional procedures 
contain discriminative stimuli that evoke 
correct academic responding at a higher 
rate. Likewise, strengthening communica- 
tive skills (a reinforcement-based 
strategy) may increase a variety of 
positive, constructive behaviors such as 
making requests, providing information, 
and protesting unwanted interactions. 
Increases in the probability of functional 
positive behaviors (whether produced by 
stimulus- or reinforcement-based 
interventions) can also potentially 
facilitate widespread changes in an 
individual’s life situation, bringing about 
improvements in social, vocational, and 
O 



educational status (i.e., lifestyle change). 
Finally, improvements in environmental 
conditions and repertoires of positive 
behavior can produce, as a side effect, 
decreases in problem behavior. In sum, 
increases in positive behavior, lifestyle 
change, and subsequent decreases in 
problem behavior define the core of PBS 
with respect to the dependent variables 
(Horner, Dunlap et al., 1990). 

Research Questions 

Having defined PBS in terms of its core 
independent and dependent variables, we 
can now pose the research questions that 
provide the structure for this review. 

How Widely Applicable Is PBS? 

The answer to this question will contrib- 
ute information concerning whether PBS 
interventions are applicable across 
gender, a broad age range, diagnosis, level 
of retardation, and type of problem 
behavior. Also, the answer will make clear 
who implements PBS (intervention 
agent) and where it takes place (interven- 
tion setting). 

In What Ways Is the Field 
Evolving? 

The answer to this question will contrib- 
ute information concerning the trends 
that have taken place over the 12-year 
period covered by the synthesis. These 
trends involve a consideration of changes 
across time in the volume of literature 
published as well as the types of interven- 
tions used, problem behaviors treated, 
assessments carried out, factors identified 
as maintaining the problem behavior, 
type of systems change, intervention 
agents and settings involved (ecological 
validity), breadth of intervention effects, 
and judgments of outcome and social 
validity made by significant others (e.g., 
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consumers such as parents, teachers, job 
coaches). The analysis of trends also 
provides information as to whether the 
field as a whole is progressing toward a 
more widespread adoption of what are 
regarded as best practices. 

How Effective Is PBS? 

Answers to this question provide critical 
information used to compare the 
effectiveness of the entire category of 
stimulus-based intervention with the 
entire category of reinforcement-based 
intervention. We also present information 
concerning changes in effectiveness when 
the categories are combined with one 
another and with non-PBS strategies that 
are frequently a part of a multicomponent 
approach to intervention. 

What Factors Modulate the 
Effectiveness of PBS? 

Answers to this question provide 
information concerning how demo- 
graphic, assessment, systems change, and 
ecological validity variables affect 
outcomes, that is, how these variables 
modulate the effectiveness of PBS-based 
intervention. 

How Responsive Is the PBS 
Literature to the Needs of Con- 
sumers (Nonresearchers)? 

Given that this question is typically not 
included in a research synthesis, we first 
discuss the rationale for its inclusion. The 
gap between the knowledge produced by 
research and the needs of consumers of 
research has prompted increasing 
national concern (Bruy^re, 1993; Carnine, 
1997; Fuchs & Fuchs, 1990; Hess & 
Mullen, 1995; Hoshmand & Polkinghorne, 
1992; Huberman, 1990; Kaufman, Schiller, 
Birman, & Coutinho, 1993; Lather, 1986; 
T ^^,Weintraub,& Safer, 1997). A 




number of factors converge, related to 
PBS, to escalate this national concern: 

(a) trends of deinstitutionalization and 
community inclusion for students with 
severe disabilities (Braddock, Hemp, 
Fujiura, Bachelder, & Mitchell, 1990), 

(b) the mandate of the Vocational 
Rehabilitation Act to conduct consumer- 
responsive research (S. Rep. No. 102-357, 
1992), and (c) the 1997 Amendments to 
the Individuals With Disabilities Educa- 
tion Act that strongly emphasize require- 
ments for functional behavior assessment 
and behavioral intervention in dealing 
with issues pertaining to student 
discipline (Individuals With Disabilities 
Education Act, 1997). 

Answers to the question related to 
the responsiveness of the PBS literature 
contribute information concerning the 
degree to which the research literature is 
usable and accessible from consumers’ 
perspectives (Carnine, 1997). We identify 
and examine in light of the database 
priority concerns in the consumer litera- 
ture. The gap between consumer concerns 
and the database contribute heuristic 
information concerning the formulation 
of a future research agenda and the 
delineation of roles related to research 
translation, dissemination, and use. 

The Structure of the 
Research Synthesis 

In Chapter 2, “Methods,” we (a) opera- 
tionally define the demographic, assess- 
ment, intervention, and outcome 
variables pertinent to the database, 

(b) explicate literature search strategies 
and eligibility (inclusion/exclusion) 
criteria, and (c) describe data collection 
and measurement methods. 

Chapter 3, “Results,” is structured 
around the first four research questions. It 
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begins with summary descriptive 
statistics on the demographic variables 
that characterize the literature included, 
thereby clarifying whether PBS is widely 
applicable across various demographic 
characteristics, problem behaviors, and 
intervention agents and settings. This 
section is followed by a presentation of 
data dealing with trends across time. 

Then we present data on intervention 
effectiveness, followed by data on 
variables that modulate effectiveness. 

In Chapter 4, “Discussion,” we first 
deal with the issue of potential biases in 
the literature retrieved. Then we develop a 
number of generalizations that can be 



inferred from the database on demo- 
graphics, assessment, interventions, and 
outcomes. Within this chapter, we discuss 
the impact of the results on assessment 
and intervention practices. The last 
section outlines where the major gaps in 
knowledge are and offers a plan for 
addressing these gaps. This section draws 
on the database to address the final 
research question, namely, how respon- 
sive the PBS literature has been to the 
needs of consumers (nonresearchers). 

In the final two chapters of the 
review, we summarize the major findings 
and provide a list of recommendations for 
advancing the field of PBS. 
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In this chapter, we (a) provide operational 
definitions of the four categories of 
variables, (b) explicate the literature 
search and eligibility criteria, and 

(c) describe the data collection methods. 

Operational Definitions 

Each article identified was scored with 
respect to four categories of variables: 

(a) demographics, (b) assessment 
practices, (c) intervention strategies, and 

(d) outcome measures. 

Demographics 

The following demographic variables 
were scored: (a) the year in which the 
article was published (1985-1996); those 
articles listed as being in press at the time 
of the review were assigned to the year 
1996, for reasons explained later in the 
section on eligibility criteria; (b) gender 
of participants (male or female); 

(c) diagnosis (mental retardation; 
autism/pervasive developmental disabil- 
ity; mental retardation + autism/ 
pervasive developmental disability; 
mental retardation and/or autism/ 
pervasive developmental disability + 
other [e.g., anxiety disorder, motor skills 
disorder, tic disorder, etc.]); (d) age (age 
in years rounded to the nearest whole 
number); (e) level of mental retardation 
(profound, severe, moderate, mild); 

(f) type of problem behavior (aggression, 
self-injurious behavior, property destruc- 
tion, tantrums). 

Assessment Practices 

Assessment practices are the methods 
practitioners and researchers use to 
O mine ( a) classes of problem 
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behavior, (b) antecedents that occasion 
and do not occasion problem behavior, 
and (c) variables responsible for main- 
taining problem behavior. These main- 
taining variables are often referred to as 
the function, purpose, goal, intent, 
reinforcers, or motivation of problem 
behavior, terms that are roughly synony- 
mous (Carr, 1993; Lee, 1988; Skinner, 
1974). 

The objective of assessment is to 
generate information that can be used to 
guide the selection and development of 
intervention strategies. 

There are three categories of 
assessment: indirect observation 
(sometimes referred to as informal 
observation), direct observation (some- 
times referred to as formal observation), 
and functional analysis (Lennox & 
Miltenberger, 1989; O’Neill et al., 1997a & 
b; Sturmey, 1994). Indirect or informal 
observation involves assessment strate- 
gies in which information about problem 
behavior (B), its antecedents (A), and 
consequences (C) are gathered indirectly 
(via informants) through the use of 
interviews (e.g., Carr et al., 1994; O’Neill 
et al., 1997 a & b), questionnaires (e.g, 
Durand & Crimmins, 1992), rating scales 
(Aman, Singh, Stewart, & Field, 1985), 
setting-event inventories that focus on 
broad contextual variables such as daily 
schedules and health status (e.g., Gardner 
& Sovner, 1994; O’Neill et al., 1997a & b), 
or anecdotal observations. Direct or 
formal observation involves direct 
measurement through the use of A-B-C 
data sheets (e.g., Bijou et al., 1 968), 
scatterplots that document temporal 
correlations between problem behavior 
and specific situations (Touchette, 
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MacDonald, 8c Langer, 1985), and time 
sampling and/or frequency counts (e.g., 
Lalli, Browder, Mace, 8c Brown, 1993; 
O’Neill et al., 1997a 8c b). Functional 
analysis involves the systematic (experi- 
mental) manipulation of the variables 
thought to control problem behavior and 
is carried out to test hypotheses about 
motivation (Carr, 1994). Assessments 
may be repeated over time if circum- 
stances change, warranting further 
investigation of motivational hypotheses 
(Mace, 1994; Vollmer, Marcus, 8c LeBlanc, 
1994). 



The product of these assessments is 
a statement concerning the problem 
behavior, controlling antecedents, and 
maintaining consequences (motivation). 
There are four commonly identified 
motivational categories: attention, escape, 
tangibles/activities, and sensory rein- 
forcement (Carr, 1977; Iwata, Dorsey, 

Slifer, Bauman, 8c Richman, 1982; Wiesler, 
Hanson, Chamberlain, 8c Thompson, 

1985). Sometimes problem behavior 
(a) functions to secure attention , 
nurturance, and comfort from others 
(Carr 8c McDowell, 1980; Lovaas, Freitag, 
Gold, 8c Kassorla, 1965; Martin 8c Foxx, 
1973); (b) helps individuals escape or 
avoid difficult, boring, or arduous tasks 
and other aversive situations (Carr 8c 
Newsom, 1985; Carr, Newsom, 8c Binkoff, 
1976, 1980; Patterson, 1982); (c) helps 
provide the individual with access to 
desirable tangible items and preferred 
activities (Derby et al., 1992; Durand 8c 
Crimmins, 1988); or (d) generates sensory 
reinforcement in the form of visual, 
auditory, tactile, and even gustatory 
stimulation (Favell, McGimsey, 8c Schell, 
1982; Rincover, Cook, Peoples, 8c Packard, 
1979). A given problem behavior may 
have more than one function (Day, 

Horner, 8c O’Neill, 1994; Haring 8c 
O My, 1990; Iwata et al., 1982). It 
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should be noted that other sources of 
motivation, such as social avoidance 
(Taylor 8c Carr, 1992), have been identi- 
fied. Indeed, it has been hypothesized that 
there may be as many as 16 different 
motives for problem behavior (Reiss 8c 
Havercamp, 1997). However, the empirical 
base demonstrating these additional 
sources of motivation is, as yet, too small 
for review purposes. 

Summary 

Using the terms just delineated, we scored 
those aspects of assessment practices that 
corresponded to the following questions: 

(a) Was there an assessment of function 
(yes/no)? (b) What assessment strategy 
was used (informal observation, formal 
direct observation, functional analysis)? 
(c) What functions were identified 
(attention, escape, tangibles/activities, 
sensory)? (d) Was the assessment 
repeated over time? (e) Was the assess- 
ment information subsequently used to 
design an intervention? 

Intervention Strategies 

Within the theme of intervention 
strategies, we scored articles with respect 
to (a) intervention category, 

(b) systems change, and (c) ecological 
validity. 

Intervention Categories 

As noted previously, there are two 
categories of PBS intervention: those 
designed to make positive behavior more 
probable by (a) remediating deficient 
environmental conditions (stimulus- 
based intervention) and (b) remediating 
deficient behavior repertoires 
(reinforcement-based intervention). 

The literature reviewed demonstrated a 
large number of variations associated 
with each category of intervention but, 



Methods 



typically, only a small number of cases 
associated with any particular variation. 

To further clarify the defining properties 
of the two generic categories that served 
as the basis for our data coding, we 
delineate here the characteristics of some 
of the many variations identified in the 
literature. 

Stimulus-Based Intervention 
From a functional standpoint, deficient 
environments provide too few stimuli 
that support positive behavior and too 
many that support problem behavior. 
Thus, one functional theme unites all 
variations in this category: Environmen- 
tal repair is attempted through the 
manipulation of stimuli that are proximal 
and distal to the behaviors of interest. The 
structural nuances that distinguish each 
variation are less important than the 
functional utility shared by all in modify- 
ing the environment to promote positive 
behavior. To illustrate the breadth of 
stimulus-based intervention, we here 
describe some of the more salient 
variations noted in the literature. 

Interspersal training. In the literature 
this strategy is variously referred to as 
interspersal training (Horner, Day, 
Sprague, O’Brien, & Heathfield, 1991 ), 
behavioral momentum (Mace, Hock, et 
al., 1988), pretask requesting (Singer, 
Singer, & Horner, 1987), task variation 
(Dunlap & Koegel, 1980; Winterling, 
Dunlap, & O’Neill, 1987), and embedding 
(Carr et al., 1976). The essence of these 
procedures is to present a stimulus (e.g., a 
difficult task demand such as “clean up 
your toys”) known to be discriminative 
for problem behavior (e.g., aggression) 
within the context of stimuli (e.g., easy 
demands such as “give me a hug”) known 
to be discriminative for nonproblem 
behavior (e.g., complying with the request 
O ) . These stimulus changes result in 



the formerly problematic stimulus (i.e., 
“clean up your toys”) now evoking 
cooperation rather than aggression. 

Expansion of choice. This strategy 
involves presenting the individual with a 
number of choice stimuli (options) 
related to a wide variety of activities and / 
or tasks, and permitting the individual to 
express a preference (choose) among the 
options. Research suggests that expand- 
ing choices can be an effective way to 
reduce problem behavior (Bannerman, 
Sheldon, Sherman, & Harchik, 1990; 
Dunlap, dePerczel et al., 1994; Dyer, 
Dunlap, & Winterling, 1990; Koegel, Dyer, 
& Bell, 1 987; Vaughn & Horner, 1997). 

Curricular modification. The essence of 
this strategy is to identify the aversive 
features of task stimuli that evoke escape- 
motivated problem behavior, and then to 
minimize or eliminate those features. 
Dunlap, Kern-Dunlap, Clarke, and 
Robbins (1991) demonstrated the efficacy 
of this procedure by altering features such 
as task length, task outcomes, and clarity 
of instructions, the modification of which 
was correlated with decreases in problem 
behavior. 

Manipulation of setting events. 

Another intervention from the generic 
stimulus-based category involves the 
manipulation of setting events. Setting 
events are broad contextual variables 
(often involving distal stimuli) that alter 
the relationship between discriminative 
stimuli and responses (Bijou & Baer, 
1961). Setting events include (a) physical 
factors such as environmental enrich- 
ment (Horner, 1980), (b) biological 
factors such as drugs (Thompson et al., 
1991) and illness (Carr & Smith, 1995), 
and (c) social factors such as the presence 
versus absence of specific people 
(Touchette et al., 1985) and the sequenc- 
ing of interpersonal activities (Brown, 
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1991). Setting events often serve as 
establishing operations (Michael, 1982), 
that is, as factors that momentarily change 
the reinforcing or aversive properties of 
response consequences, thereby influenc- 
ing the probability of constructive 
behavior as well as problem behavior. 

Reinforcement-Based Intervention 
A deficient positive-behavior repertoire 
makes it difficult (or impossible) for an 
individual to meet his or her needs (i.e., 
access preferred reinforcers), which, in 
turn, increases the level of frustration 
(i.e., maximizes episodes of extinction), 
thereby leading to problem behavior; if an 
individual’s current repertoire of 
nonproblem (positive) behavior is 
ineffective in gaining reinforcers but his 
or her problem behavior is effective, then 
problem behavior will become more 
probable. From a functional standpoint, 
the presence of positive behaviors 
compete with and/or make problem 
behavior unnecessary because the 
positive behaviors themselves provide 
alternative avenues for accessing valued 
reinforcers. 

There are many variations of 
reinforcement-based interventions. One 
common theme unites all the variations: 
They target specific behaviors or classes 
of behaviors for consistent, systematic 
reinforcement. 

By way of illustrating this broad 
generic category, we here describe several 
of the many variations of reinforcement- 
based interventions noted in the litera- 
ture. Note that one procedure, differential 
reinforcement of other behavior (DRO), 
despite its name, is not an example of 
reinforcement-based intervention given 
the criteria we are using. DRO involves 
delivering reinforcement contingent on 
the nonoccurrence of the target problem 
k ~ u avior for a prespecified period of time 



(Vollmer & Iwata, 1992). No positive 
behavior(s) is (are) explicitly targeted for 
reinforcement, which is why some 
researchers prefer to call the procedure 
“differential reinforcement of not 
responding” or “zero responding” (Poling 
8c Ryan, 1982; Zeiler, 1970). Indeed, some 
have argued that the DRO procedure can 
be viewed as a form of punishment, 
because frequent display of problem 
behavior results in repeated omission of 
positive reinforcers, an aversive event 
(Rolider 8c Van Houten, 1990). 

Functional communication training. 

This intervention involves teaching an 
individual a specific communicative 
response that serves the same function 
(functional equivalence) as the problem 
behavior it is intended to replace (Carr 8c 
Durand, 1985). In illustration, a func- 
tional analysis indicates that a young girl 
with autism becomes self-injurious in the 
presence of negative academic feedback 
because of a history of reinforcement for 
such behavior (i.e., self-injury results in 
termination of the putatively aversive 
instructional situation). Following the 
functional analysis, the teacher makes 
changes. She prompts the girl to request 
“help” in response to negative feedback 
(e.g., the teacher says, “No, that’s not the 
right answer”; then the teacher provides a 
prompt, “Say, ‘Help me, please’”). If the 
new communicative response is more 
efficient (Horner 8c Day, 1991) at termi- 
nating the aversive events associated with 
the task than self-injury (i.e., communi- 
cation requires less effort to escape from 
the negative feedback than self-injurious 
behavior), then the problem behavior is 
likely to decrease (Bird, Dores, Moniz, 8c 
Robinson, 1989; Carr 8c Durand, 1985; 
Day, Rea, Schussler, Larsen, 8c Johnson, 
1988; Durand 8c Carr, 1991, 1992; Wacker 
etal., 1990). 
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Self-management. Self-management 
involves any of three component skills: 
self-monitoring, self-evaluation, and self- 
reinforcement. Self-monitoring involves 
teaching an individual to discriminate 
appropriate versus inappropriate 
behaviors and to describe each of them 
(e.g.,‘T made my bed” versus “I bit my 
hand”). Self-evaluation consists of 
labeling a behavior as desirable versus 
undesirable. For example, after making 
the bed, the individual might be taught to 
say, “I did a good job.” In contrast, after 
self-biting, the individual might be taught 
to say, “I didn t do a good job” Self- 
reinforcement consists of teaching the 
individual to deliver reinforcers (e.g., 
praise, tangibles) to him- or herself 
following a positive self-evaluation but 
not after a negative self-evaluation. In 
sum, the individual is taught to positively 
reinforce desirable response alternatives 
to the problem behavior. Studies suggest 
that instruction in self-management can 
reduce or eliminate disruptive, aggressive, 
and self-injurious behaviors (Gardner, 
Cole, Berry, & Nowinski, 1983; Koegel, 
Koegel, Hurley, & Frea, 1992). 

Differential reinforcement of alterna- 
tive behavior (DRA). This intervention 
involves reinforcing those behaviors that 
are topographically different from the 
targeted problem behavior. The new 
behaviors thus serve as alternatives to the 
problem behavior. In an early demonstra- 
tion of DRA, Hall, Lund, and Jackson 
(1968) reinforced (with teacher attention) 
the “study” behavior of elementary school 
students, while applying extinction to 
their disruptive behavior. As the alterna- 
tive (study) behaviors were strengthened, 
there was a concomitant decrease in 
problem behavior (disruption). 
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Non-PBS (Environmentally Based) 
Intervention 

There were many instances in the 
literature in which PBS interventions 
were combined with non-PBS (environ- 
mentally based) interventions. These 
latter interventions were defined as those 
for which the primary goal was the 
reduction of problem behavior through 
the direct application of procedures 
reactive to the display of problem 
behavior. Of the many variations of non- 
PBS procedures, three are described that 
illustrate the reactive nature of non-PBS 
(in contrast to the proactive nature 
of PBS). 

The first procedure, differential 
reinforcement of other behavior (DRO), 
has already been discussed. In brief, DRO, 
applied during a bout of problem 
behavior, involves delivering reinforce- 
ment contingent on the nonoccurrence of 
problem behavior for a prespecified 
period of time. A second procedure, 
extinction, involves withdrawing the 
reinforcer that maintains the problem 
behavior each time that the problem 
behavior occurs. The status of extinction 
as a non-PBS procedure is debatable. In a 
later section, we discuss the possibility, 
based on conceptual and pragmatic 
considerations, that extinction might also 
be viewed as a key aspect of PBS. A third 
procedure, timeout, involves the with- 
drawal of all positive reinforcement for a 
fixed time period following the occur- 
rence of problem behavior. This proce- 
dure is considered a form of punishment. 

Systems Change 

PBS involves systems change, not just 
change in the individual who displays 
problem behavior. Both stimulus-based 
intervention and reinforcement-based 
intervention potentially result in changes 
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in how other people respond to the 
person with disabilities and how the 
environment is organized. We examined 
these two aspects of systems change. 

Behavior Change on the Part of 
Significant Others 
We determined whether people other 
than the person with disabilities were 
required to alter aspects of their behavior 
as part of the intervention. In illustration, 
consider functional communication 
training. If an individual with disabilities 
was taught to request help in response to 
a difficult task, then the support person 
(e.g., a parent or teacher) would be 
expected to respond to the request by 
providing help; the support person was 
required to change his or her behavior in 
response to change in the behavior of the 
person with disabilities. If an investigator 
explicitly noted such a requirement, we 
scored this aspect of systems change as 
being present. 

Broad Environmental Reorganization 
and Restructuring 
We also determined whether broad 
reorganization and/or restructuring of 
the environment (Emerson et al., 1994) 
was reported as part of the intervention 
approach. These environmental variables 
included any variation of the following: 
systematic personnel changes; alterations 
in the scheduling of activities; provision 
of supported employment; provision of 
new, enriching community activities; 
provision of respite services; friendship 
facilitation; provision of additional staff; 
physical alteration of the home and/or 
school setting; addition (and/or removal) 
of individuals with disabilities to/from 
specific classrooms and/or group homes; 
and provision of choices all day long (i.e., 
not just in selected circumstances). If an 
investigator systematically included one 
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or more of the preceding variables as part 
of an intervention, this aspect of systems 
change was scored as being present. 

Ecological Validity 

PBS is not intended to be a laboratory- 
based demonstration or analog but, 
rather, a strategy for dealing with 
problem behavior in all pertinent natural 
contexts. We documented this aspect of 
PBS intervention by examining (a) who 
carried out the intervention (the inter- 
vention agent), (b) where the intervention 
took place (intervention setting), and 
(c) whether the intervention was imple- 
mented in all contexts in which problem 
behavior was noted to occur (all relevant 
contexts). 

Intervention Agent 
A distinction can be made between 
intervention agents who would normally 
be expected to be the primary support 
people/caregivers in a particular commu- 
nity setting (hereafter referred to as 
typical intervention agents) and those 
who would not normally be involved 
(hereafter referred to as atypical interven- 
tion agents). In the home setting, the 
typical intervention agent would be a 
parent or other close relative of a child or 
adolescent; in the school setting, a 
teacher; in a group home setting, direct 
care staff; in supported living, a 
housemate; and, in the workplace, a job 
coach or designated fellow employee. We 
defined all such individuals, in these and 
other relevant community settings, as 
typical intervention agents. In contrast, 
atypical intervention agents included 
psychologists, behavior specialists, 
researchers, and others who would not be 
expected to provide support on a day-to- 
day basis under normal circumstances. 

In our synthesis, the involvement of 
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typical intervention agents reflects high 
ecological validity; the involvement of 
atypical agents reflects low validity. 

Intervention Setting 
A distinction can be made between living 
environments that are considered 
normative for an individual of a given age 
(hereafter referred to as typical settings), 
and those not considered normative 
(hereafter referred to as atypical settings). 
According to this criterion, typical 
settings include the home, integrated 
school, group home/own home, job site, 
neighborhood, and a variety of commu- 
nity settings (e.g., those related to 
recreational activities, shopping, eating, 
and entertainment). Using the same 
criterion, atypical settings include 
segregated schools, psychiatric wards/ 
hospitals, medical clinics, state institu- 
tions, and sheltered workshops. With 
respect to the concept of PBS, interven- 
tions taking place in typical settings 
reflect high ecological validity, while 
those taking place in atypical settings 
reflect low validity. 

Intervention in All Relevant Contexts 
Context has two dimensions: temporal 
and situational. For scoring purposes, the 
relevant context for carrying out an 
intervention for problem behavior 
included all the time periods for which 
the problem was reported to occur, and 
all the situations for which it was reported 
to occur. In illustration, a teacher might 
report that problem behavior occurred 
throughout the school day and across 
many different situations (e.g., during 
gym, lunch time, reading, group circle, 
dismissal, boarding the bus). An investi- 
gator might respond to the teacher s 
referral by removing the child from the 
classroom and conducting 20-minute 
intervention sessions in a special tutorial 
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room. Because these sessions do not cover 
the entire time period or array of 
situations for which the teacher reported 
the presence of problem behavior, we 
scored the implemented intervention as 
not having occurred in all relevant 
contexts. In contrast, a teacher might 
report that problem behavior occurred 
exclusively during the 20-minute daily 
gym session because the motor activities 
involved were singularly aversive to the 
individual. To bring about improved 
behavior, an investigator might instruct 
the teacher, in the gym setting, to alter the 
curriculum during the 20-minute session. 
In this case, the intervention session 
covers the entire time period in the 
situation identified by the teacher. This 
intervention would be scored as having 
occurred in all relevant contexts. 

Because PBS interventions are 
intended to deal with problem behavior 
whenever and wherever it naturally 
occurs, the first intervention described 
would reflect low ecological validity; the 
second would reflect high validity. 

Summary 

Using the terms just delineated, we scored 
those aspects of an intervention that 
corresponded to the following questions: 
(a) Was the intervention stimulus-based, 
reinforcement-based, non-PBS based 
(yes/no for each)? (b) Systems change: 

Did the intervention involve change on 
the part of significant others and/or 
broad environmental reorganization and 
restructuring (yes/no for each)? 

(c) Ecological validity: What type of 
intervention agent was involved (typical/ 
atypical)? What type of intervention 
setting was involved (typical/atypical)? 
Did intervention occur in all relevant 
contexts (yes/no)? 
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Outcome Measures 

Because PBS involves the multidimen- 
sional remediation of deficient context, 
the outcomes of the approach are likewise 
multidimensional. We examined seven 
outcome measures: (a) positive behavior, 
(b) problem behavior, (c) stimulus 
generalization, (d) response generaliza- 
tion, (e) maintenance, (f) lifestyle change, 
and (g) social validity. 

Positive Behavior 

PBS intervention involves the use of 
strategies designed to make socially 
desirable responses (positive behaviors) 
more probable. Therefore, we examined 
whether positive behaviors did indeed 
increase following intervention. For each 
article we scored changes in positive 
behaviors from baseline to intervention. 

In illustration, consider a reinforcement- 
based procedure such as functional 
communication training. Because the 
procedure teaches specific communica- 
tive alternatives to problem behavior, one 
would expect to see increases in the level 
(i.e., frequency, percentage) of this 
alternative behavior following interven- 
tion. When data on communicative 
responses were reported in an article, 
we scored them. In principle, each 
reinforcement-based procedure should be 
associated with an increase in one or 
more types of positive behavior. • 

Likewise, stimulus-based procedures 
should promote positive behavior, by 
altering features of the environment. In 
illustration, consider a procedure such as 
interspersal training. When a difficult 
task demand (known to be discriminative 
for problem behavior) is interspersed 
among stimuli known to be discrimina- 
tive for cooperation, one would expect an 
increase in the level of the latter positive 
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behavior. Again, in our synthesis we 
scored all data reflecting changes in 
positive behavior. 

Problem Behavior 

We measured another key outcome: . 
whether an intervention produced a 
decrease in problem behavior. Interven- 
tion effects were always measured in 
terms of percentage reduction in problem 
behavior relative to baseline (using the 
calculation method described later in this 
review). 

Stimulus Generalization 

We measured stimulus generalization, 
defined as the'degree to which interven- 
tion effects transferred from the original 
intervention situation to other situations 
involving new intervention agents, 
settings, and tasks. Stimulus generaliza- 
tion thus referred to a behavior change 
that occurred in spite of the fact that no 
planned intervention occurred in the new 
situation. We also measured the degree to 
which decreases in problem behavior 
generalized to new situations; this was 
also measured in terms of percentage 
reduction in problem behavior relative to 
baseline. 

Response Generalization 

We measured response generalization, 
defined as the degree to which interven- 
tion effects transferred from the initial 
target(s) of intervention to other aspects 
of the individuals behavior repertoire not 
targeted for intervention. Let’s say an 
intervention targets ameliorating self- 
injurious behavior via communication 
training. The outcome may also demon- 
strate that aggressive behavior decreased 
and social play increased even though 
these two behaviors were not the focus of 
intervention. The desirable -side effects of 
intervention (i.e., the decrease in 
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aggression, and the increase in social 
play) constitute response generalization. 
Again, these effects were measured as 
percentage change from baseline. 

Maintenance 

Maintenance was defined as the degree to 
which intervention effects lasted over 
time (intervention durability). Mainte- 
nance was further defined as involving 
any of the following: (a) those data 
collected only after the specific interven- 
tion had been completely terminated; 

(b) data collected after the intervention 
had been modified (but not terminated) 
in some substantive way, such as a 
decrease in the number of formal 
intervention sessions per unit of time, a 
decrease in the number of intervention 
components in effect (i.e., intervention 
fading), or a gradual reduction in the 
involvement of the intervention agent 
(i.e., the agent decreases the amount of 
time given to intervention implementa- 
tion). We measured maintenance effects 
as percentage reduction from baseline; 
when available, effects were noted at these 
specific follow-up periods: 1 to 5 months, 
6 to 12 months, 13 to 24 months, and 25 
months or more. 

Lifestyle Change 

Because the purpose of PBS is not simply 
to reduce the level of problem behavior, 
but also to enable individuals to live more 
normalized lives, a key outcome measure 
relates to lifestyle change. Positive lifestyle 
change was defined as increased engage- 
ment in normative social, vocational, 
family, recreational, and academic 
activities. Lifestyle change effects were 
measured, when available, as percentage 
increase from baseline. 



Social Validity 

Wolf (1978) argued that the impact of 
interventions cannot be gauged solely by 
objective measures. Unless significant 
others (e.g., parents, teachers, job 
coaches, friends, members of the 
community) perceive the intervention 
and its effects to be worthwhile, the 
intervention would be judged as inad- 
equate. Considering this, social validity 
was also measured as an outcome. 

We examined articles to determine 
whether rating scales were reported that 
tapped three critical dimensions of social 
validity: feasibility, desirability, and 
effectiveness. The generic feasibility 
dimension involved any variant of the 
question, “Would you be able to use this 
intervention strategy?” The generic 
desirability question involved any variant 
of the question, “Would you be willing to 
use this intervention strategy?” The 
effectiveness dimension was subdivided 
into (a) effectiveness with respect to 
reduction in problem behavior, and 
(b) effectiveness with respect to lifestyle 
change. Effectiveness with respect to 
problem behavior involved any variant of 
the question, “Does this intervention 
strategy reduce problem behavior to a 
level that is acceptable to you?” Effective- 
ness with respect to lifestyle change 
involved any variant of the question, 
“Does this intervention strategy make a 
difference in the lifestyle of the individual 
involved in terms of increasing opportu- 
nities to live, work, go to school, recreate, 
and socialize with typical peers and 
significant others in typical community 
settings?” 
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Summary 

Using the terms just delineated, we scored 
those aspects of outcome measurement 
related to the following questions: 

(a) Were data available on positive 
behaviors resulting from the intervention 
(yes/no)? If so, specify the baseline and 
intervention data for each positive 
behavior, (b) Were data available on 
reduction in problem behavior following 
intervention (yes/no)? If so, specify the 
baseline and intervention data for each 
type of problem behavior reported. 

(c) Regarding stimulus generalization, 
were anecdotal observations available 
(yes/no)? Were direct observation data 
available (yes/no)? If direct observation 
data were available, specify the baseline 
and intervention data for problem 
behavior, (d) Regarding response 
generalization, were anecdotal observa- 
tions available (yes/no)? Were direct 
observation data available (yes/no)? If 
direct observation data were available, 
specify the baseline and intervention data 
for socially appropriate behavior and 
problem behavior, (e) Regarding mainte- 
nance, were data on reduction in problem 
behavior noted at the following specified 
follow-up periods: 1 to 5 months, 6 to 12 
months, 13 to 24 months, 25 months or 
more (yes/no for each)? If direct observa- 
tion data were available, specify the 
baseline and intervention data. 

(f) Regarding lifestyle change, was a 
lifestyle change considered (anecdotally 
or formally) as a goal of the study (yes/ 
no)? Was there a formal intervention to 
improve lifestyle (yes/no)? Was there 
measured success in producing a lifestyle 
change (yes/no)? If direct observation 
data were available on lifestyle change, 
specify the baseline and intervention 
data, (g) Regarding social validity, was 
there a generic feasibility question (yes/ 
’* r as there a generic desirability 



question (yes/no)? Was there a generic 
question regarding effectiveness with 
respect to reduction in problem behavior 
(yes/no)? Was there a generic question 
regarding effectiveness with respect to 
lifestyle change (yes/no)? For each of the 
preceding questions, specify the pre- 
and postintervention data for each scale. 

Literature Search and 
Eligibility Criteria 

We established initial selection criteria. 
Then we conducted a literature search 
using these criteria. Finally, we applied 
exclusion criteria to eliminate those 
articles that did not meet desired 
methodological standards. 

Initial Selection Criteria 

There were six criteria that guided the 
initial selection of articles from the 
literature. First, an article had to have 
been published between 1985 and 1996. 
All articles accessible to us by the cutoff 
date of December 31,1 996, were consid- 
ered. Some 1996 journals had delayed 
publication dates in which the final issue 
for 1996 was released three to four 
months past the cutoff date. To compen- 
sate for the potential loss of relevant 1996 
articles as well as to achieve the most up- 
to-date review, we included a small 
number (6) of in press articles that we had 
obtained prior to the cutoff date. These 
were classified as 1996 articles. (However, 
it should be noted that these 1996 articles 
were no longer in press by the time this 
review was completed and, therefore, are 
cited in the reference list as 1997 articles.) 

Second, in an attempt to ensure high 
standards, only articles published in peer- 
reviewed journals were considered. Non- 
peer-reviewed manuscripts were not 
considered in the analysis in part to help 
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ensure a uniformly high standard of 
experimental rigor, and in part because 
single-subject research tradition relies on 
the judgment of anonymous peer 
reviewers to confirm the presence of a 
functional relationship. The inclusion of 
non-peer-reviewed studies (e.g., disserta- 
tions, clinical trials) would have imposed 
uncontrolled sources of error, because 
each study would have required a 
surrogate peer review (by us) prior to 
inclusion to document the presence of 
experimental control. 

Third, the article had to have been 
published in English. 

Fourth, with respect to diagnosis, we 
examined all variations of DSM-III, DSM- 
III-R, DSM-IV, and AAMR classifications 
related to mental retardation, autism, and 
pervasive developmental disorder, either 
as a primary or secondary diagnosis. 
Thus, relevant dual diagnoses (e.g., 
anxiety disorder of childhood with 
mental retardation) were also retained. 

Fifth, with respect to topography, the 
following types of problem behaviors 
were examined: self-injury, aggression, 
property destruction, and tantrums. 

Sixth, with respect to intervention, 
all variations of stimulus- and rein- 
forcement-based intervention, as defined 
earlier, were included. 

Literature Search 

We began by hand-searching all relevant 
education, psychology, and medical 
journals listed in four previous reviews 
that had included a consideration of PBS 
(Carr et al., 1990; Didden et al., 1997; 
Scotti et al., 1991; Scotti et al., 1996). The 
articles gleaned from this initial process 
produced references to additional 
research articles, review papers, books, 
BnnL- chapters, and newsletters. These 




reference trails were, in turn, pursued. 
Additional reference trails were generated 
when the following abstract and index 
services were searched by crossing the 
disability diagnoses with the problem 
behavior topographies: Child Development 
Abstracts and Bibliography, Current 
Contents/Social and Behavioral Sciences , 
ERIC, MEDLINE, Psychological Abstracts, 
PsychINFO, PsychLIT, PsychSCAN/MR, and 
the Social Science Citation Index. 

We also requested information on 
intervention for problem behavior from 
organizations having a stake in providing 
services for people with disabilities. The 
National Information Center for Children 
and Youth With Handicaps provided us 
with their list of 33 stakeholder organiza- 
tions that included The Association for 
Persons With Severe Handicaps, The ARC 
(formerly the Association for Retarded 
Citizens), Autism Society of America, 
Council for Exceptional Children, and The 
National Down Syndrome Society. 

Finally, we requested information 
from leading researchers (14), asking 
them to send us their published and in 
press papers dealing with the issue of 
problem behavior. We defined “leading 
researcher” as any individual having at 
least three published articles related 
to PBS. 

Using the initial selection criteria and 
search methods just explicated, we 
identified 216 articles from 36 journals. 

Exclusion Criteria 

To ensure the highest quality database for 
subsequent analyses, we applied a number 
of methodological exclusion criteria to the 
initial sample of 216 articles. The applica- 
tion of these criteria resulted in a final 
sample of 109 articles that were included, 
and 107 that were excluded. (It should be 
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Table 1. Journals Examined and Number of 
Articles Included and Excluded 




Journal 


No. of articles 


No. of articles 


included 


excluded 


Adult Foster Care Journal 


0 


1 


American Journal of Medical Genetics 


1 


0 


American Journal on Mental Retardation 


0 


9 


(formerly American Journal of Mental Deficiency) 
Analysis and Intervention in Developmental Disabilities 


2 


3 


Applied Research in Mental Retardation 


0 


1 


Augmentative and Alternative Communication 


1 


0 


Australia and New Zealand Journal of Developmental Disabilities 1 


3 


Behavior Modification 


6 


5 


Behavior Therapy 


4 


0 


Behavioral Disorders 


2 


0 


Behavioral Interventions 


3 


3 


Behavioral Residential Treatment 


3 


13 


Behaviour Research and Therapy 


0 


1 


Behavioural Psychotherapy 


0 


5 


Child and Family Behavior Therapy 


1 


1 


Education and Training in Mental Retardation 


3 


7 


(formerly Education 8c Training of the Mentally Retarded) 
Education and Treatment of Children 


3 


1 


Exceptional Parent 


0 


1 


Journal of Applied Behavior Analysis 


46 


14 


Journal of the Association for Persons With Severe Handicaps 


10 


7 


Journal of Autism and Developmental Disorders 


2 


5 


Journal of Behavioral Education 


2 


0 


Journal of Behavior Therapy and Experimental Psychiatry 


4 


2 


Journal of Consulting and Clinical Psychology 


1 


0 


Journal of Developmental and Physical Disabilities 


4 


5 


Journal of Intellectual Disability Research 


1 


1 


Journal of the Multihandicapped Person 


2 


0 


Journal of Visual Impairment and Blindness 


0 


1 


Mental Handicap Research 


0 


6 


Mental Retardation 


0 


5 


Research in Developmental Disabilities 


3 


2 


School Psychology Review 


3 


0 


Special Services in the Schools 


1 


0 


Teaching Exceptional Children 


0 


1 


The Irish Journal of Psychology 


0 


1 


Topics in Early Childhood Special Education 


0 


3 


Total 


109 


107 



Nnte- The citation information for each included article 



IS 

ERIC 



in asterisk^). 
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Table 2. Criteria Used to Exclude Articles From the Database 


Exclusion criterion 


Number of articles rejected 


Absence of data 


23 . . 


Inadequate design 


28 ■ 


Group design 


12 


Pooled data 


33 


Insufficient baseline data 


31 


Insufficient intervention data 


7 



Note: Some articles were rejected for more than one reason. 



noted that some of the contributions 
made by the excluded studies will be 
discussed later in this publication.) Table 
1 shows the breakdown of the two 
samples across the 36 journals. 

Table 2 lists the six exclusion criteria 
used to select the final sample and the 
number of articles excluded for failing to 
meet a given criterion. Some articles were 
excluded for more than one reason, and 
thus the total is greater than 107. 

There were 23 articles excluded 
because no data were reported (absence 
of data). These articles took the form of 
narrative case reports and extended 
anecdotes. 

Twenty-eight articles were excluded 
because of inadequate design. Specifically, 
these articles involved empirical case 
reports that employed an AB design (i.e., 
a baseline (A) condition was followed by 
an intervention (B) condition). AB 
designs do not meet the internal validity 
criteria enunciated in standard method- 
ology texts on single-subject research 
designs (e.g., Hersen & Barlow, 1976). In 
contrast, articles based on multiple 
baseline, reversal, and withdrawal 
designs, all meet these criteria (e.g., 
Hersen & Barlow, 1976), and were 
O lined for analysis. 

ERIC 



A small number of articles (12) used 
legitimate group designs involving a 
comparison between experimental and 
control groups. But these articles reported 
only group means, making it impossible 
to determine how any one individual 
responded to an intervention. Because all 
of our subsequent analyses depended on 
having individual data, the results of 
these articles could not be integrated into 
our final database. Further, 6 of these 
articles either failed to report critical data 
on problem behavior or pooled multiple 
measures in a way that made it impos- 
sible to retrieve data pertaining to 
problem behavior per se. 

An additional 33 articles were 
excluded because they reported only 
pooled data; these articles did not use 
legitimate group designs but, rather, 
reported averaged pre/post measures for 
a group of participants with no control 
group. Again, the absence of data on any 
one individual precluded the possibility 
of integrating these data with the results 
obtained from other studies in which 
individual data were retrievable. 

Thirty-one articles used acceptable 
single-subject designs but were nonethe- 
less excluded because they reported fewer 
than three baseline points (often only 1 
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point). Our descriptive analyses (de- 
scribed later) required at least three 
baseline data points. 

Likewise, seven articles were 
excluded because of insufficient interven- 
tion data (i.e., fewer than 3 intervention 
data points). 

Data Collection Methods 

How Intervention Effects 
Were Measured 

One of the central issues we address is the 
impact of PBS on reducing problem 
behavior. We always measured interven- 
tion effects in terms of percentage 
reduction of problem behavior relative to 
baseline. Throughout the text, we will use 
the terms percentage reduction and 
suppression measure as synonyms. Many 
of the articles used a reversal design; that 
is, the intervention condition alternated 
several times with the baseline condition. 
When this type of design was used, we 
deemed the final, rather than earlier, 
intervention conditions most important, 
because the critical issue to be addressed 
concerned how well an individual was 
doing at the end of intervention. Addi- 
tionally, because intervention frequently 
produced a steady downward trend in the 
level of problem behavior, the overall 
mean for an intervention condition could 
actually underrepresent the final effect. 

To minimize this difficulty, the mean of 
the last three intervention data points was 
used so that a judgment could be made 
concerning the degree of participant 
improvement at the termination of 
intervention. In sum, intervention effects 
were measured as the percentage 
reduction in problem behavior from the 
last three sessions of baseline compared 



against the final three intervention 
sessions of the final intervention 
condition. 

Frequently, a single participant 
received more than one type of interven- 
tion. For example, a participant might 
first be exposed to functional communi- 
cation training, then choice, and finally 
interspersal training. That is, the person 
would have received one reinforcement- 
based intervention followed by two 
stimulus-based interventions, providing 
an opportunity to examine three out- 
comes. When this situation arose, we 
coded the data separately for each of the 
three outcomes. In contrast, if a partici- 
pant had received choice in three different 
phases of a reversal design, we coded only 
the final outcome because, in this situation, 
the same stimulus-based intervention 
(choice) was simply repeated in each 
phase. So for a given participant, an 
outcome was defined as the data associ- 
ated with each unique (nonrepetitive) 
variation of a reinforcement-based and/or 
stimulus-based intervention. Table 3 
shows the number of outcomes per 
participant. As can be seen, out of a 
total of 230 participants, 145 (63%) 
produced a single outcome, while 85 
(37%) produced more than one outcome. 
The total number of outcomes pooled 
across participants was 366. 

To calculate percentage reduction in 
problem behavior (suppression measure), 
it was necessary to estimate the data from 
each article, point by point, thus generat- 
ing the baseline and intervention data 
pertaining to the 366 outcomes. We 
measured, from the published tables and 
figures, the last three baseline points 
prior to intervention and, as noted 
previously, the last three intervention 
points of intervention. The intervention 
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Table 3. Relationship Between Outcomes and Participants 


Number of 


Number of outcomes 


Total number of 


participants 


per participant 


outcomes 


145 


1 


145 


60 


2 


120 


13 


3 


39 


6 


4 


24 


2 


5 


10 


2 


6 


12 


0 


7 


0 


2 


8 


16 



Note: The total number of participants was 230, and the total number of outcomes was 366. 



mean was subtracted from the baseline 
mean, divided by the latter, and then 
multiplied by 100 to yield the percentage 
reduction from baseline. In illustration, 
consider a participant whose baseline 
frequency (last 3 data points) of aggres- 
sive behavior was 10, 12, and 14 aggres- 
sive acts, and whose intervention 
frequency (last 3 data points) of such 
behavior was 5, 1, and 0 acts. The baseline 
mean was thus 12, and the intervention 
mean 2. The percentage reduction from 
baseline was therefore (12 - 2) + 12 x 100 
= 83.3%. This method was used for data 
reported as a frequency or as a percentage 
of time samples observed. In a handful of 
cases (6), data were reported as latency to 
problem behavior (e.g., 10 s elapsed from 
the beginning of an observation session 
to the first instance of aggression; 
therefore, the latency to aggression was 
10 s). In this case only, the data were 
transformed as follows. Any reported 
latency, irrespective of magnitude, was 
scored as 1 (i.e., problem behavior 
occurred after a specific time interval). 
The absence of problem behavior was 
scored as 0. Transforming the data set to 
O 




this binary code permitted the calcula- 
tions already described. In illustration, if 
the baseline were 1,1,1 and the interven- 
tion data were 1, 0, 0, then the percentage 
reduction would be 67%. 

Recall that, in addition to evaluating 
the initial reduction in problem behavior 
following intervention, we also examined 
the data on positive behavior, stimulus 
generalization, response generalization, 
and maintenance. For these four outcome 
measures, intervention data were 
typically not reported; if they were, often 
fewer than three data points were 
available (e.g., a single data point on 
stimulus generalization might have been 
reported). To provide some indication of 
effectiveness regarding the four measures, 
we compared the three baseline data 
points with even one intervention point 
if that was all that was available. In 
illustration, if the baseline stimulus 
generalization data were 12,14, and 
16 self-injurious acts and the intervention 
stimulus generalization datum was 
2 self-injurious acts, then the percentage 
reduction was (14 - 2) 14 x 100 = 

85.7%. 
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How Reliability Was 

Measured 

Three types of reliability were computed 
related to (a) adherence to initial 
selection and exclusion criteria; 

(b) agreement on scoring of categorical 
and continuous data; and (c) data entry 
(keystroke errors). 

With respect to the initial selection 
and exclusion criteria, recall that the 
initial selection criteria produced 216 
articles, and that the exclusion criteria 
resulted in 109 articles being retained, 
and 107 articles being excluded. We 
randomly chose 50 articles from the 109 
that were retained, and 50 articles from 
the 107 that were excluded. We then gave 
these 100 articles to one of the coau- 
thors who was not involved in the initial 
selection. This coauthor was asked to 
perform two rating tasks: (a) to apply 
the six initial selection criteria (de- 
scribed earlier) to the 100 articles and 

render a judgment as to whether the 

articles met these criteria, and 
(b) to apply the six exclusion criteria 
(described earlier) to the 100 articles 
and render a judgment as to whether the 
articles met these criteria. The rater 
agreed with the originalcoder’s decision 
that all 100 articles met the initial 
selection criteria (100% reliability), and 
that the 50 articles that had been 
excluded by the original coder should 



indeed have been excluded as per the 
criteria used (100% reliability). 

With respect to the categorical and 
continuous data, four coders scored these 
data from the 109 articles. Again, one of 
the coauthors not involved in the original 
scoring randomly selected 7 articles 
originally scored by each of the four 
coders and recoded all of the categorical 
and continuous data from this sample of 
28 articles. Recall that the continuous data 
were based on the following variables: 
positive behavior, problem behavior, 
stimulus generalization, response gener- 
alization, maintenance, and lifestyle 
change. All the remaining variables (e.g., 
age, gender, problem behavior topography, 
etc.) were categorical. For the continuous 
data, the Pearson product-moment 
correlation, based on point-by-point 
reliability, was +0.99 (p = .000). For the 
categorical data, Kappa values (Cohen, 
1960) ranged from .82 to 1.00. Landis 8c 
Koch (1977) characterized Kappa values 
greater than .75 as representing excellent 
agreement beyond chance. 

With respect to data entry (keystroke 
errors), each of the four coders reentered 
data for seven randomly selected articles 
that they had previously scored. There was 
a total of 81,921 keystrokes across the four 
coders, of which 94 differed from the 
original entries, yielding an error rate of 
only 0.11%. 
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RESULTS 



As noted before, 109 published articles 
that met all the inclusion and exclusion 
criteria produced data on 230 participants 
involving 366 outcomes. We now present 
these data with respect to the research 
questions posed earlier in this review. 

How Widely 
Applicable is PBS? 

Table 4 displays the data on the character- 
istics of participants involved in PBS 
interventions as well as data on the type of 
intervention agent and type of interven- 
tion setting associated with the various 
participants. In a small number of cases, 
investigators failed to specify pertinent 
information (e.g., no diagnosis was given) 
or the description given was ambiguous 
(e.g., the participant was said to be 
“retarded” but no level of retardation was 
noted). Therefore, the numbers reported 
represent the percentage of the database 
for which each characteristic was ad- 
equately specified. 

Approximately twice as many males 
as females were involved in PBS interven- 
tions. Although PBS was applied across the 
entire age range, preschool children were 
least likely to receive this type of interven- 
tion followed, in increasing order, by 
adolescents and adults (i.e., those 20 years 
of age and over); elementary-school-age 
children were most likely to receive 
intervention. The latter two age categories 
each accounted for one third of the 
participants. 

About half the participants were 
diagnosed as having mental retardation, 
and one tenth as having autism. The 
remaining participants had combined 




diagnoses of retardation and/or autism, 
frequently accompanied by additional 
diagnoses (e.g., seizure disorder, brain 
damage). With respect to level of retarda- 
tion, one third of the participants were 
functioning in the profound range, and 
another third in the severe range. The 
remaining participants were equally 
divided between the moderate and mild 
range. 

The data on type of problem behavior 
showed that about one third of the 
participants displayed self-injurious 
behavior, and almost a quarter displayed 
aggression. Property destruction and 
tantrums were exhibited by only a small 
percentage of the participants. However, 
various combinations of the preceding 
four types of problem behavior were 
shown by fully one third of the partici- 
pants. 

PBS was a little more likely to be 
implemented by atypical intervention 
agents (e.g., psychologists, researchers) 
than typical agents (e.g., parents, teach- 
ers), although in a very small number of 
cases, both atypical and typical agents 
combined their efforts! In contrast, by a 
ratio of 2:1, interventions were more likely 
to occur in atypical settings (e.g., segre- 
gated schools, medical clinics) than 
typical settings (e.g., home, community, 
integrated schools). 

In What Ways is 
the Field Evolving? 

The field is evolving, as indicated by the 
ways in which selected aspects of the data 
set changed over time. These trends are 
presented next. 
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Table 4. Characteristics of Participants, 
Intervention Agents, and Settings 



Characteristic 


Number of cases 




Gender 


Male 


150 


67.6 


Female 


72 


32.4 


Age in years 


0-4 


27 


11.7 


5-12 


78 


33.9 


13-19 


53 


23.0 


>20 


72 


31.3 


Diagnosis 


Mental retardation (MR) 


114 


51.4 


Autism (Aut) 


25 


11.3 


MR + Aut 


34 


15.3 


MR +/or Aut plus other diagnoses 


49 


22.1 


Level of retardation 


Mild 


32 


16.7 


Moderate 


36 


18.8 


Severe 


64 


33.3 


Profound 


60 


31.3 


Type of problem behavior 


Aggression 


51 


22.2 


Self-injurious behavior 


78 


33.9 


Property destruction 


6 


2.6 


Tantrums 


11 


4.8 


Combinations 


84 


36.5 


Intervention agent 


Typical 


99 


44.2 


Atypical 


120 


53.6 


Both typical and atypical 


5 


2.2 


Intervention setting 


Typical 


79 


34.3 


Atypical 


150 


65.2 


Both typical and atypical 


1 


.4 



Note: Number of cases does not always sum to 230 due to missing data. 
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Size of the Database 

Figure 1 shows the data on the total number 
of articles, participants, and outcomes 
across 4-year blocks of time. In this and 
all subsequent figures, the numbers 
directly above each bar graph refer to the 
raw data, and the height of the bars refers 
to the data converted into percentages. In 
illustration, consider the data on articles 
published. Of the 109 articles, 24 (22%) 
were published between 1985 and 1988, 

29 (26.6%) between 1989 and 1992, and 
56 (51.4%) between 1993 and 1996. There 
is a substantial increase in the number of 
PBS articles published over time. It 
should be noted there were no patterns to 
the exclusion of articles across years. That 
is, our criteria did not result in more 
articles being excluded from early years 
than from recent years. Similar increases 
are seen for both the number of partici- 
pants and the number of outcomes. 



Demographics 

Figure 2 shows the percentage of partici- 
pants, by gender, in each 4-year block of 
time. In this and all subsequent figures 
(as was evident from Table 4), the data 
presented are for those cases on which 
information was available. Thus, although 
there were 230 participants in all, the 
gender for 8 of them was not identified 
(missing data), so percentages were 
computed for 222 participants rather 
than 230. Again, for this and subsequent 
figures, there was generally an increase in 
the number of participants over time, 
because greater numbers of articles were 
published over time. The important 
information, therefore, concerns the 
relative proportion of males to females 
over time. In each block of time, males 
were approximately twice as numerous as 
females. 
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Figure 3 shows the percentage of 
participants, by age, in each 4-year block 
of time. Generally, elementary-school-age 
children (5-12 years old), adolescents (13- 
19 years old), and adults (20 years and 
older) were well represented in each time 
block. Preschool children were generally 
less well represented. 



Figure 5 shows the percentage of 
participants by level of mental retarda- 
tion. Participants associated with 
profound and severe levels were generally 
more numerous in each block of time 
than those with moderate and mild levels 
of retardation. 




Figure 4. Percentage of total participants (N = 222) by diagnosis in each 4-year block of time. 
Numbers over each bar graph refer to the raw data. MR = mental retardation; Aut = autism; 
Both = MR + Aut; Plus = MR &/or Aut plus other diagnoses. 



Figure 4 shows the percentage of 
participants, by diagnosis, over time. 
Participants.with a diagnosis of mental 
retardation were most numerous in each 
time block with no clear trends evident 
for the other diagnoses. 



Figure 6 shows the percentage of 
participants by types of problem behav- 
ior. In the first time block, aggression, 
self-injurious behavior, and combinations 
of problems were equally numerous. Over 
time, self-injurious behavior and 
combinations gradually became more 
numerous than aggression. Property 
destruction and tantrums remained at 
low levels across blocks of time. 
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Figure 5. Percentage of total participants (N - 192) by level of retardation in each 4-year block 
of time. Numbers over each bar graph refer to the raw data. P = profound; S = severe; Mod = 
moderate; M = mild. 




Figure 6. Percentage of total participants (N = 230) by type of problem behavior in each 4-year 
block of time. Numbers over each bar graph refer to the raw data . Agg = aggression; SIB = self 
irrwnnus behavior; PD = property destruction; T = tantrums; Comb = combinations of types. 
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Figure 7. Percentage of total participants (N = 224) by type of intervention 
agent in each 4-year block of time. The numbers over each bar graph refer to the 
raw data. Typ = typical intervention agents; Atyp = atypical intervention 
agents; Both = both typical and atypical agents. 



Figure 7 shows the percentage of 
participants involved with different types 
of intervention agents. Except for 1989- 
1992, typical and atypical agents were 
involved to the same degree. The involve- 
ment of both typical and atypical agents 
for the same participant was rare. 



Figure 8 shows the percentage of 
participants by type of intervention 
setting. Although participants were most 
likely to be seen in atypical settings in 
each block of time, the gap between the 
use of atypical and typical settings 
steadily narrowed over time. The use of 
both atypical and typical settings for the 
same participant was rare. 
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Figure 8. Percentage of total participants (N = 230) by type of intervention 
setting in each 4-year block of time. The numbers over each bar graph refer to the 
raw data. Typ = typical intervention settings; A typ = atypical intervention 
settings; Both = both typical and atypical settings. 
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Figure 9. Percentage of total outcomes (N = 366) by prior assessment in each 
4-year block of time. The numbers over each bar graph refer to the raw data. 
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Figure 10. Percentage of total outcomes (N = 266) by type of assessment in each 4-year block of 
time. Numbers over each bar graph refer to the raw data. 10 = indirect observation; DO = direct 
observation; FA = functional analysis; Comb = combinations of 2 or more types of assessment. 



Assessment 

Figure 9 shows the percentage of out- 
comes for which a prior assessment was 
(yes) or was not (no) carried out. There 
was a dramatic increase, over time, in the 
proportion of outcomes that were 
associated with a prior assessment 
compared to those that were not. 

Figure 10 shows the percentage of 
outcomes for which a specific type of 
assessment was completed (i.e., indirect 
observation only, direct observation only, 
functional analysis only, and combina- 
tions of the preceding types of assess- 
ment). Over time, combined assessments 
and functional analysis became propor- 
tionately greater than indirect and direct 
observation. 

Additional analyses permitted an 
on c wer to the question of whether 




assessment practices varied with respect 
to type of intervention agent, type of 
intervention setting, and whether 
intervention was subsequently carried out 
in all relevant contexts. With respect to 
type of intervention agent, for 257 
outcomes it was possible to determine 
both the type of agent involved and the 
type of assessment involved. We con- 
trasted the assessment data generated 
from formal functional analysis (FA- 
based) with the data based on less formal 
indirect and direct observation (not FA- 
based). FA-based procedures consisted of 
FA alone or FA used in combination with 
indirect or direct observation. Non-FA- 
based procedures excluded the use of FA 
and involved indirect and/or direct 
observation only. There were 200 out- 
comes that.were FA-based. Of these, 
typical agents were associated with 73 

49 



35 



Chapter 3 



outcomes (36.5% of the total), atypical 
agents with 124 outcomes (62.0% of the 
total), and both typical and atypical 
agents with 3 outcomes (1.5% of the 
total). There were 57 outcomes that were 
not FA-based. Of these, typical agents 
were associated with 41 outcomes (71.9% 
of the total), atypical agents with 14 
outcomes (24.6% of the total), and both 
typical and atypical agents with 2 
outcomes (3.5% of the total). In sum, FA- 
based outcomes were more likely to be 
associated with atypical agents, and non- 
FA-based outcomes were more likely to be 
associated with typical agents. 

With respect to type of intervention 
setting, for 266 outcomes it was possible 
to determine both the type of setting 
involved and the type of assessment 
involved. There were 203 outcomes that 
were FA-based. Of these, typical settings 
were associated with 64 outcomes (31.5% 
of the total), and atypical settings with 
139 outcomes (68.5% of the total). There 
were 63 outcomes that were not FA-based. 
Of these, 32 outcomes (50.8% of the total) 
were associated with typical settings, 30 
outcomes (47.6% of the total), with 
atypical settings, and 1 outcome (1.6% of 
the total) with both typical and atypical 
settings. In sum, FA-based outcomes were 
more likely to be associated with atypical 
settings, and non-FA-based outcomes 
were equally likely in typical and atypical 
settings. 

With respect to intervention in all 
relevant contexts, for 237 outcomes it was 
possible to determine whether interven- 
tion in all relevant contexts subsequently 
occurred, and the type of assessment 
preceding the intervention. There were 
179 outcomes that were FA-based. Of 
these, 28 outcomes (15.6% of the total) 
were associated with intervention in all 
relevant contexts, but 151 outcomes 
O 



(84.4% of the total) were not. Fifty-eight 
outcomes were not FA-based. Of these, 29 
outcomes (50% of the total) were 
associated with intervention in all 
relevant contexts, and an equal number 
were not. In sum, FA-based outcomes 
were less likely to be associated with 
intervention in all relevant contexts, and 
non-FA-based were associated half the 
time with intervention in all relevant 
contexts and half the time without such 
intervention. 

Overall, then, the use of functional 
analysis as an assessment tool is more 
closely associated with atypical agents, 
atypical settings, and a failure to inter- 
vene in all relevant contexts. In contrast, 
non-FA-based assessment is more closely 
associated with typical agents but is not 
differentially associated with type of 
setting or the presence versus absence of 
intervention in all relevant contexts. 

Figure 1 1 shows the percentage of 
outcomes associated with each type of 
behavioral motivation for the 250 
outcomes in which motivation (function) 
could be determined. In each block of 
time, escape predominated over other 
single motivations, and the degree to 
which this was the case increased over 
time. There was also some indication of a 
proportionate increase over time in the 
number of outcomes associated with 
multiple motivations (combinations of 
motivations). 

Finally, there were only 1 1 outcomes 
(out of the total of 266) for which 
assessment was repeated over time. 
Between 1 985 and 1 988, 0 outcomes 
involved repeated assessments; between 
1989 and 1993, there was 1 such outcome 
(.3% of the total); and between 1993 and 
1996, there were 10 (3.8% of the total). 
Repeated assessment became more 
common over time, though the sample 
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Type of Motivation 



Figure 1 1. Percentage of total outcomes (N = 250) by type of motivation in each 4-year block of 
time. Numbers over each bar graph refer to the raw data. Att = attention ; Esc — escape, Tan — 
tangible; Sen = sensory; Comb = 2 or more types of motivation. 



size was very small. Our additional 
analyses allowed an answer to the 
question of when assessment was most / 
least likely to be repeated. Specifically, for 
246 outcomes, it was possible to deter- 
mine the type of setting and intervention 
agent for which the assessment was not 
repeated. With respect to settings, 
assessment was not repeated for 79 
outcomes occurring in typical settings 
(32.1% of the total), 166 outcomes 
occurring in atypical settings (67.5% of 
the total), and 1 outcome occurring in 
both a typical and atypical setting (.4% of 
the total). With respect to intervention 
agents, assessment was not repeated for 
106 outcomes occurring with typical 
agents (43.1% of the total), 136 outcomes 
occurring with atypical agents (55.3% of 
the total), and 4 outcomes occurring with 
O l typical and atypical agents ( 1 .6% of 
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the total). For 1 1 outcomes, it was 
possible to determine the type of setting 
and intervention agent for which the 
assessment was repeated. With respect to 
settings, assessment was repeated for 10 
outcomes occurring in typical settings 
(90.9% of the total), 1 outcome occurring 
in atypical settings (9.1% of the total), 
and 0 outcomes occurring in both a 
typical and atypical setting. With respect 
to intervention agents, assessment was 
repeated for 8 outcomes occurring with 
typical agents (72.7% of the total), 2 
outcomes occurring with atypical agents 
(18.2% of the total), and 1 outcome 
occurring with typical and atypical 
agents (9.1% of the total). In sum, then, 
assessment was most likely to be repeated 
with typical agents and settings, and 
most likely not to be repeated with 
atypical agents and settings. 

51 



37 



Chapter 3 



E 

o 
o 
*— • 

s 

O 



eS 

Vm 

O 

0> 

8P 

■4— » 

s 

a 

,p 

Oh 



40 



30 



ST/RF 



20 



10 - 



1993-1996 

91 



1985-1988 1989-1992 

54 



14 



50 50 



14 



65 



Non-PBS 

1989-1992 

79 

1985-1988 
58 



1993-1996 
127 



24 



24 



H u- -C 
o 
oa 



25 



53 



CO d 



o 

OQ 



iH tij 



o 

03 



# 2 #2 



<*> o 
.cu 



&o o 
£ 2 



Type of Intervention 



J gUr t} 2 ' Perc f nta 8 e of total outcomes (N = 366) by type of intervention in each 4-year block of 
“ The " umbers °\ er each ^r graph refer to the raw data. ST = stimulus-based intervention; 
RF- reinforcement-based intervention; Both = combined stimulus-based and reinforcement- 
based intervention; Non-PBS = nonpositive behavior support procedure was/was not (Yes/No) part 
oj the intervention. J " 



Intervention 

Figure 12 shows the percentage of 
outcomes associated with various types 
of interventions. The relative proportion 
of stimulus-based versus reinforcement- 
based intervention reversed over time. 
Initially, reinforcement-based interven- 
tion predominated (1985-1988); then the 
two types of intervention were equally 
applied (1989-1992); and finally 
stimulus-based intervention predomi- 
nated (1993-1996). Combined interven- 
tion (i.e., both stimulus-based and 
reinforcement-based) showed no clear 
trend relative to the other two types. 
Additional analyses permitted an answer 
to the question of when combined 



intervention versus noncombined 
intervention (i.e., stimulus-based only or 
reinforcement-based only) was most/least 
likely to occur. Specifically, there were 42 
outcomes for which combined interven- 
tion occurred. Of these, 26 (61.9% of the 
total) were carried out by typical agents 
and only 15 (35.7% of the total) were 
carried out by atypical agents. There were 
324 outcomes for which noncombined 
intervention occurred. Of these, there 
were 9 outcomes for which type of agent 
could not be determined, leaving a 
database of 3 1 5 outcomes. For the 
remaining cases (315 outcomes), the 
pattern previously described was 
reversed. Specifically, only 128 (40.6% of 
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the total) were carried out by typical 
agents, but 181 (57.5% of the total) were 
carried out by atypical agents. There were 
7 outcomes involving both typical and 
atypical agents together (2.2% of the 
total). 

Data were also available on type of 
setting. Of the 42 outcomes for which 
combined intervention occurred, 22 
(52.4% of the total) were carried out in 
typical settings, and 19 (45.2% of the 
total) were carried out in atypical 
settings. In 1 outcome combined inter- 
vention was carried out in both typical 
and atypical settings (2.3% of the total). 

In sharp contrast, for the 324 outcomes 
involving noncombined intervention, 104 
(32.1% of the total) were carried out in 
typical settings and 220 (67.9% of the 
total) were carried out in atypical 
settings. In sum, then, combined inter- 
vention was most likely to be conducted 
by typical agents and in typical settings, 
and noncombined intervention was most 
likely to be conducted by atypical agents 
and in atypical settings. 

One final comparison concerns the 
relationship between type of intervention 
and its use in all relevant contexts. There 
were 322 outcomes for which it was 
possible to determine both the type of 
intervention used and whether it 
occurred in all relevant contexts. 

There were 285 outcomes associated with 
noncombined intervention. Of these, 50 
(17.5% of the total) were associated with 
intervention in all relevant contexts, and 
235 (82.5% of the total) were not. There 
were 37 outcomes associated with 
combined intervention. Of these, 26 
(70.3% of the total) were associated with 
intervention in all relevant contexts, and 
1 1 (29.7% of the total) were not. In sum, 
combined intervention was most likely to 
in all relevant contexts, whereas 



noncombined intervention was least 
likely to occur in all relevant contexts. 

Figure 12 also displays the data for 
outcomes associated with interventions 
that included a non-PBS component (yes) 
in addition to a PBS component versus 
outcomes not associated with interven- 
tions that included a non-PBS component 
(no) (i.e., the intervention had only PBS 
components). Consider the first block of 
time (1985-1988). There were 24out- 
comes associated with interventions that 
included a non-PBS component, but 58 
outcomes associated with interventions 
that did not include a non-PBS compo- 
nent. This pattern of results, namely, the 
predominance of outcomes for which PBS 
was the only component of the interven- 
tion package, was evident across each 
block of time. 

Figure 13 shows the percentage of 
outcomes associated with different types 
of systems change. With respect to 
whether significant others were/were not 
(yes/no) required to change their 
behavior as a component of intervention, 
it is clear that the presence of such change 
predominated in all three time blocks; the 
degree of this predominance dramatically 
increased in the final time block. With 
respect to whether environmental 
reorganization was/was not (yes/no) a 
component of intervention, it is clear that 
the presence of such change was rare 
over time. 

Figure 14 shows the percentage of 
outcomes in which intervention was/was 
not (yes/no) carried out in all relevant 
contexts. Over time, there was a dramatic 
increase in the number of outcomes 
associated with a failure to intervene in 
all relevant contexts, but only a modest 
increase in those associated with 
intervention in all relevant contexts. 
Additional analyses allowed us to 
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Type of Systems Change 



Figure 13. Percentage of total outcomes (N = 366) by type of systems change in each 4-year block 
of time. The numbers over each bar graph refer to the raw data. SIG = significant others were 
required to change their behavior; ENV = environmental reorganization was undertaken. 



determine when intervention was most/ 
least likely to be carried out in all relevant 
contexts. Specifically, for 241 outcomes, it 
was possible to determine the type of 
setting and type of intervention agent for 
which intervention was not (no) carried 
out in all relevant contexts. With respect 
to settings, intervention was not carried 
out in all relevant contexts for 70 out- 
comes occurring in typical settings 
(29.0% of the total), and 171 outcomes 
occurring in atypical settings (71.0% of 
the total). With respect to agents, such 
intervention was not carried out for 82 
outcomes occurring with typical agents 
(34.0% of the total), 157 outcomes 
occurring with atypical agents (65.1% of 
the total), and 2 outcomes occurring with 
both typical and atypical agents (1.0% of 
the total). For 72 outcomes, it was 
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possible to determine the type of setting 
and type of intervention agent for which 
intervention was (yes) carried out in all 
relevant contexts. With respect to settings, 
intervention was carried out in all 
relevant contexts for 42 outcomes 
occurring in typical settings (58.3% of 
the total), 29 outcomes occurring in 
atypical settings (40.3% of the total), and 
1 outcome occurring in both typical and 
atypical settings (1.4% of the total). With 
respect to agents, such intervention was 
carried out for 62 outcomes occurring 
with typical agents (86. 1% of the total), 5 
outcomes occurring with atypical agents 
(6.9% of the total), and 5 outcomes 
occurring with both typical and atypical 
agents (6.9% of the total). In sum, then, 
intervention in all relevant contexts was 
most likely to occur with typical agents 



54 



Results 




Figure 14. Percentage of total outcomes (N = 322) by intervention in all relevant 
contexts in each 4-year block of time. The numbers over each bar graph refer to 
the raw data. Yes = intervention was carried out in all relevant contexts; No - 
intervention was not carried out in all relevant contexts. 



and settings, and most likely not to occur 
with atypical agents and settings. 

Outcomes 

Figure 15 shows the percentage of 
outcomes associated with diverse 
measures of generalization. Typically, data 
were based on small numbers of out- 
comes. The percentage of outcomes for 
stimulus generalization showed no trend 
over time, but that for response generali- 
zation showed a slight increase over time. 
The percentage of outcomes reported for 
maintenance for 1 to 5 months’ duration 
rose steadily over time, but that for 6 to 12 
months, and 13 to 24 months showed no 
trend. Likewise, no trends were seen for 
rcentage of outcomes associated 



with a stated goal of lifestyle change, nor 
was there a trend with respect to the use 
of planned intervention intended to 
produce lifestyle change. Finally, the 
direct measurement of lifestyle change 
following intervention showed a small 
increase over time. 

Figure 16 shows the percentage of 
outcomes associated with diverse 
measures of social validity. In all cases, 
data were based on very small numbers 
of outcomes. The percentage of outcomes 
for which there was a feasibility question 
showed a modest increase over time, but 
percentage for the desirability question 
showed no trend. Likewise, we noted a 
small increase for the question concern- 
ing the acceptability of the level of 
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Figure 15. Percentage of total outcomes (N= 366) by type of generalization in each 4-year block 
of time. The numbers over each bar graph refer to the raw data. SG = stimulus generalization; RG 
= response generalization; Ml, M6, and M 13 = maintenance after 1-5 months, 6-12 months, and 
13-24 months respectively; LG = stated goal of lifestyle change; LI = lifestyle change intervention 
implemented; LS = lifestyle change success measured. 




Type of Social Validation 



Figure 16. Percentage of total outcomes (N = 366) by type of social validation in each 4-year 
block of time. The numbers over each bar graph refer to the raw data. Feas = feasibility; 

O = desirability; Acc = acceptability; Life = lifestyle change. 
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Figure 17. Number of outcomes for which 
changes in positive behavior were measured. 
The bar graph above represents the data for 
outcomes measured in terms of changes in 
frequency (N = 43), and the bar graph below 
represents the data for outcomes measured in 
terms of changes in percentages (N = 122). 
Numbers over each bar graph refer to the 
number of outcomes associated with each 
category of change. 
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problem-behavior reduction, but there 
was no trend concerning whether 
significant others perceived the interven- 
tion to have produced meaningful 
lifestyle change. 

How Effective is PBS? 
Changes in Positive Behavior 

PBS intervention involves strategies 
designed to make socially desirable 
responses (positive behaviors) more 
probable. We identified data on changes 
in positive behavior for 165 of the 366 
outcomes (45.1% of the total). Because 
the baseline for positive behavior was, in 
a substantial number of cases, 0, it was 
not possible to compute a percentage 
increase in positive behavior relative to 
baseline (because division by 0 is 
impermissible). Therefore, as Figure 17 
shows, difference scores based on the raw 
data were used. For example, if the data 
were reported as frequencies (as they 
were in 43 cases), we simply subtracted 
the mean of the last three baseline points 
(typically 0) from the mean of the last 
three intervention points. In illustration, 
if the baseline mean was 0 communica- 
tive acts, and the intervention mean was 
30 communicative acts, then the differ- 
ence score (i.e., increase in communica- 
tive acts) was 30. In Figure 1 7, this data 
outcome would contribute to the bar 
graph category >10 & 50. We used the 
same formula for data reported as 
percentages (as they were in 122 cases). 

In illustration, if the baseline mean was 
10% intervals of cooperative behavior, 
and the intervention mean was 90% 
intervals of cooperative behavior, then the 
difference score (i.e., increase in percent- 
age of intervals containing cooperative 
behavior) was 80%. Figure 17 displays 
both the frequency and percentage data 
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on positive behavior. In either category, 
frequency or percentages, interval data 
are not directly comparable. For example, 
a frequency increase of 50 communica- 
tive acts is not “twice as good” as a 
frequency increase of 25 cooperative acts. 
The two types of behavior are not 
equivalent. Nonetheless, the data in 
Figure 17 are presented to provide 
information to answer the general 
question of whether positive behaviors 
did indeed increase following PBS. The 
figure shows clearly that there were only a 
handful of cases in which the level of 
positive behavior remained the same or 
decreased following intervention. 
Overwhelmingly, for both the frequency 
and percentage data, positive behaviors 
increased following the use of PBS, albeit 
the degree of increase varied widely from 
modest to substantial. 

Distribution of Outcome 
Effectiveness for PBS 
Interventions 

Table 5 shows the distribution of outcome 
effectiveness in terms of percentage 



reduction in problem behavior from 
baseline. Across all PBS interventions 
(i.e., irrespective of whether they were 
stimulus-based or reinforcement-based), 
approximately two-thirds of the outcomes 
(68.0%) were associated with substantial 
reductions in problem behavior of 80% or 
more from baseline levels. A similar 
pattern was seen when the total database 
for intervention was subdivided: out- 
comes that were stimulus-based versus 
those that were reinforcement-based. 
Specifically, 66.5% of the outcomes that 
were stimulus-based and 71.6% of the 
outcomes that were reinforcement-based 
demonstrated 80% or more reduction in 
problem behavior from baseline levels. In 
contrast, across the intervention catego- 
ries, a minimal number of outcomes were 
associated with small reductions (i.e., less 
than 20% reduction from baseline levels). 
For approximately 6 to 8% of the out- 
comes, depending on the type of inter- 
vention, an increase in problem behavior 
was noted. It would be interesting to 
know whether there are specific variables 
that predict the few instances in which 
PBS produced minimal or negative 



Table 5. Distribution of Outcome Effectiveness: 
Reduction in Problem Behavior 



Frequency of outcomes by percentage reduction in problem behavior 

Increase in 





100% 


90- 

99% 


80- 

89% 


60- 

79% 


40- 

59% 


20- 

39% 


0 - 

19% 


problem 

behavior 


Total 

outcomes 


All PBS-based outcomes 


97 


92 


60 


51 


23 


10 


6 


27 


366 


ST-based outcomes 


44 


52 


35 


31 


12 


6 


5 


12 


197 


RF-based outcomes 


67 


48 


36 


25 


12 


4 


2 


17 


211 



Note: PBS = any intervention that had an ST and/or RF component; ST = any stimulus-based 
intervention (i.e., ST with/without RF, with/without non-PBS); RF = any reinforcement-based 
intervention (i.e., RF with/without ST, with/without non-PBS). Thus, outcomes associated with ST 
and RF in combination would appear once under ST-based and again under RF-based. 
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effects. But the articles reviewed provided 
insufficient information to make such a 
determination. A reasonable working 
hypothesis is that intervention failures are 
more a function of systems than of 
techniques. Thus, uncooperative staff, too 
many staff changes, lack of respite for 
parents, and insufficient time on the part 
of teachers are all plausible examples of 
the types of systemic factors likely to 
impact negatively on intervention 
outcomes, a point taken up again later. At 
present, the existence of unfavorable 
outcomes should serve as a prompt for 
researchers to expand their search for 
those systems variables that appear to 
influence outcome. 



Success Rates for PBS 
Interventions Pooled Across 

Outcomes 

Table 6 shows the success rates for 
various types of PBS interventions. A 
stringent criterion was used to define 
success: An outcome had to reflect a 90% 
or greater reduction in problem behavior 
from baseline levels to be considered a 
success. The success rate was generally 
within 5 points of 50%, irrespective of the 
type of intervention employed. 

Almost three-quarters of the 
interventions (72.1%) did not include a 
non-PBS component; however, the 
presence versus absence of a non-PBS 



Table 6. Success Rates for PBS Interventions 
Pooled Across Outcomes 



Pooled data 


Number of 
outcomes 


Percentage of 
grand total 


Number 
of successes 


Percentage of 
of successes 


All interventions (grand total) 


366 


100 


189 


51.6 


All that excluded non-PBS 


264 


72.1 


136 


51.5 


All that included non-PBS 


102 


27.9 


53 


52.0 


All single interventions 


324 


88.5 


167 


51.5 


All ST interventions 


155 


42.3 


74 


47.7 


All ST excl. non-PBS 


129 


35.2 


59 


45.7 


All ST incl. non-PBS 


26 


7.1 


15 


57.7 


All RF interventions 


169 


46.2 


93 


55.0 


All RF excl. non-PBS 


104 


28.4 


62 


59.6 


All RF incl. non-PBS 


65 


17.8 


31 


47.7 


All combined interventions 


42 


11.5 


22 


52.4 


All ST + RF that excl. non-PBS 


31 


8.5 


15 


48.4 


All ST + RF that incl. non-PBS 


• 11 


3.0 


7 


63.6 



Note: ST = stimulus-based interventions; RF = reinforcement-based interventions; excl. = 
excluding; incl. = including. Success was defined as a 90% or more reduction in problem behavior 
Sseline. 



ERIC 



53 



45 



Chapter 3 



Table 7. Types of Non-PBS Interventions Used 


Type of non-PBS 


Number of outcomes 


Extinction 


58 


DRO 


6 


Punishment 


49 


Reprimand 


4 


Forced compliance 


6 


Response cost 


8 


Overcorrection 


6 


Timeout 


7 


Brief restraint 


17 


Water mist 


1 



Note: The total number of outcomes added up to more than 102, because some outcomes were 
associated with several types of non-PBS. 



component had little effect on interven- 
tion success (i.e., each intervention type 
produced success rates close to 50%). A 
substantial majority of interventions 
(88.5%) used stimulus-based strategies 
or reinforcement-based strategies, but not 
both together. The two strategies were 
used about equally often. The success 
rates for the two were comparable (i.e., 
47.7% for stimulus-based, and 55.0% 
for reinforcement-based). Only a small 
minority of interventions (1 1.5%) 
combined stimulus-based with 
reinforcement-based strategies. These 
combined interventions also produced 
success rates of about 50%. The one 
exception to this general finding involved 
combined interventions that included one 
or more non-PBS interventions; these 
produced a 63.6% success rate. However, 
this rate was based on the smallest 
number of outcomes for any of the PBS 
intervention categories reported and may, 
therefore, reflect sampling error rather 
than a unique property of this type of 
intervention. 

O 



Because non-PBS interventions were 
associated with approximately one 
quarter (102) of the 366 outcomes, it is 
helpful to clarify the specific nature of 
these interventions. Table 7 presents a 
breakdown of the types of non-PBS 
interventions used. About half the 
outcomes were associated with the use of 
extinction and half with the use of 
punishment. A handful were associated 
with DRO. The punishment category was 
further broken down into seven proce- 
dures. Of these, only brief physical 
restraint was associated with more than 
eight outcomes. Among the seven 
punishment procedures, only water mist 
has been considered in the literature 
(Scotti et al., 1991) as a highly intrusive 
procedure, and it was associated with 
only 1 outcome of the 102 generated. 
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Table 8. Generalization Measures of Outcome Effectiveness 

Frequency of outcomes by percentage reduction in problem behavior 

Increase in 

100% 90- 80- 60- 40- 20- 0- problem Total 

99% 89% 79% 59% 39% 19% behavior outcomes 



Stimulus generalization 

outcomes 5 8 3 2 1 0 0 1 20 



Frequency of outcomes by percentage increase in appropriate behavior 

Decrease in 

100% 90- 80- 60- 40- 20- 0- appropriate Total 

99% 89% 79% 59% 39% 19% behavior outcomes 



Response generalization 

outcomes 0 1 2 3 4 1 1 1 13 



Stimulus and Response 
Generalization 

Stimulus generalization refers to the 
degree to which intervention effects 
transferred from the original intervention 
situation to other situations involving 
new intervention agents, settings, and 
tasks. Anecdotal reports of stimulus 
generalization were noted for 29 out- 
comes. Data-based reports of stimulus 
generalization were noted for an addi- 
tional 24 outcomes (i.e., in 6.6% of the 
total database). However, in 4 of these 
cases, the baseline data were inadequate 
for computation purposes (i.e., fewer than 
3 data points were reported). The 
distribution of the remaining 20 out- 
comes is shown in Table 8 (top half). As 
can be seen, in a number of cases, 
problem behavior decreased from 
baseline levels, sometimes substantially, 
in situations that were not a direct target 
of intervention. 

Response generalization refers to the 
degree to which intervention effects 







transferred from the initial behavioral 
target of intervention to other aspects of 
an individual’s behavior repertoire that 
were not targeted for intervention. In the 
articles examined, response generaliza- 
tion consisted exclusively of changes in a 
variety of appropriate behaviors (e.g., 
social skills, academics). Anecdotal 
reports of response generalization were 
noted for 25 outcomes. Data-based 
reports of response generalization were 
noted for an additional 26 outcomes (i.e., 
in 7.1% of the total database). However, in 
13 of these cases, baseline data were 
inadequate (i.e., fewer than 3 data points 
were reported), or baseline data were 
reported but postintervention data were 
not. The distribution of the remaining 13 
outcomes is shown in Table 8 (bottom 
half). As can be seen, in a number of 
cases, modest increases in appropriate 
behavior were observed relative to 
baseline, even though such behavior was 
not a target of intervention. 
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Table 9. Maintenance Measures of Outcome Effectiveness 

Frequency of outcomes by percentage reduction in problem behavior 

Increase in 

100% 90- 80- 60- 40- 20- 0- problem Total 







99% 


89% 


79% 


59% 


39% 


19% 


behavior 


outcomes 


1 - 5 months 


62 


6 


8 


14 


1 


2 


1 


5 


99 


6 - 12 months 


19 


9 


5 


5 


4 


2 


0 


0 


44 


13 - 24 months 


5 


0 


1 


0 


1 


0 


0 


0 


7 



Note: There were no maintenance outcomes reported for follow-up periods of 25 months or more. 



Maintenance 

As noted earlier, maintenance is defined 
as the degree to which intervention 
effects were documented to last over time 
(intervention durability). Table 9 displays 
the outcome data for maintenance, 
measured as percentage reduction in 
problem behavior from baseline, noted at 
specific follow-up periods, namely, at 1 to 
5 months, 6 to 12 months, 13 to 24 
months, and 25 months or more. The 
number of outcomes noted for each of the 
follow-up periods just mentioned was 99, 
44, 7, and 0, indicating a dramatic 
decrease in available data with each 
successive increase in maintenance 
duration. In general, most outcomes 
clustered at the high end of the percent- 
age reduction in problem behavior. That 
is, good maintenance effects were 
observed for a substantial majority of 
outcomes. Using a success criterion of 
90% or more reduction in problem 
behavior from baseline, the success rates 
at 1 to 5 months, 6 to 12 months, and 13 
to 24 months were 68.7%, 63.6%, and 
71.4% respectively, suggesting that 
intervention effects were quite durable, at 
least for those cases in which mainte- 
Q e outcome data were available. 



Lifestyle Change 

As the ultimate purpose of PBS is to 
enable individuals to live more normal- 
ized lives, lifestyle change is an important 
index of effectiveness. Surprisingly, the 
database for this outcome measure was 
extremely small. Lifestyle change was a 
stated intervention goal for only 24 out of 
the 230 participants in the sample (i.e., 
10.4% of the sample). A formal interven- 
tion directed specifically at improving 
lifestyle was recorded for 8 out of the 230 
participants (i.e., 3.5% of the sample). 
Finally, success in improving lifestyle was 
measured for only 6 participants (i.e., 
2.6% of the sample). Of these, anecdotal 
(nonquantified) improvement was noted 
for 4 participants. Data (percentage 
improvement from baseline) were taken 
on only 2 participants, and showed a 100% 
improvement with respect to increased 
engagement in community activities. 

Social Validity 

Social validity refers to whether signifi- 
cant others (e.g., parents, teachers, job 
coaches) perceive the intervention and its 
effects to be worthwhile. (It should be 
noted that no study involved asking 
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persons with disabilities what they 
thought about the social validity of the 
interventions that they experienced.) 
Again, various aspects of social validity 
were assessed for a very small number of 
participants. With respect to the generic 
feasibility question (e.g., “Would you be 
able to use this intervention strategy?”), 
data were available on 14 of the 230 
participants. In 8 cases, these data were 
anecdotal; teachers (for example) stated 
that they had continued to use the 
intervention over time. For the remaining 
6 participants, Likert-scale data indicated 
that whereas intervention agents thought 
they would seldom/never be able to use 
the intervention before they had been 
trained, afterwards they felt they would 
very much/always be able to use the 
intervention. 

In terms of the generic desirability 
question (e.g., “Would you be willing to 
use this intervention strategy?”), data 
were available on 12 participants. In 7 
cases, these data were anecdotal state- 
ments from support people affirming the 
desirability of the intervention used over 
all other proposed or previously at- 
tempted interventions. For the remaining 
5 participants, Likert-scale data were 
available. These data showed that, prior to 
training, support people felt they would 
choose to implement the intervention 
never/not at all; following training, they 
would choose the intervention very 
much/always. 

With respect to the generic accept- 
ability question (e.g., “Does the interven- 
tion strategy reduce problem behavior to 
a level that is acceptable to you?”), data 
were available for 29 participants. In 4 
cases, these data were anecdotal. Support 
people were interviewed and asserted 
that each of the participants had “im- 
proved.” In 25 cases, data were available 
O 




but the metric used varied greatly from 
study to study. For example, in 6 cases, 
support people were asked to rate the 
degree to which (strongly agree/agree/ 
uncertain/disagree/strongly disagree) 
they felt that the participant’s problem 
behavior had improved. In 4 of these 
cases, 65% of the support people strongly 
agreed/agreed; in the other 2 cases, 83% 
strongly agreed/agreed. In 2 cases, 
support people were asked to rate 
problem behavior on a 6-point scale 
(where 1 represented low improvement 
and 6, high improvement). In 1 case, the 
rating improved from 1.8 to 2.9; in the 
other case, from 2.2 to 4.1. In 13 addi- 
tional cases, significant others were asked 
to rate problem behavior on a 7-point 
scale (where 1 represented low improve- 
ment and 7, high improvement). For 8 of 
these cases, the ratings improved from a 
preintervention mean of 2.1 to a post- 
intervention mean of 5. 1 . For 5 of these 
cases, the postintervention mean was 5.9, 
but no preintervention data were reported, 
complicating interpretation. Finally, in 4 
cases, significant others were asked to use 
a 10-point scale (where 1 was little/no 
improvement and 10 was maximum 
improvement/no problem behavior). The 
preintervention mean was 2.1, and the 
postintervention mean was 7.2. 

For the generic lifestyle change 
question (e.g., “Does this intervention 
strategy produce effects that increase 
opportunities to live, work, go to school, 
recreate, and socialize with typical peers 
and significant others in typical commu- 
nity settings?”), data were available for 
8 participants. In 6 of these cases, lifestyle 
change was rated on a 7-point scale 
(where 1 represented little improvement 
and 7, major improvement). Across 3 of 
these cases, the preintervention mean was 
2.3, and the postintervention mean, 6.0. 

c 
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In the other 3 cases, the postintervention 
mean was 3.2, but there were no 
preintervention data, again complicating 
interpretation. In the remaining 2 cases, 
significant others were interviewed by 
phone 2 years after intervention and 
asked whether they agreed/disagreed that 
family interactions had improved as a 
consequence of intervention and that the 
child was more accepted in the commu- 
nity; 86% of the interviewees agreed with 
the preceding statement. 

What Factors 
Modulate the 
Effectiveness of PBS? 

The data on intervention effectiveness, 
just examined, make dear that stimulus- 
based and reinforcement-based interven- 
tions produced very similar outcomes. Of 
course it is possible that each type of 
intervention might, nonetheless, produce 
a unique interaction with modulator 
variables such as demographic character- 
istics or type of assessment used. 

However, from a clinical perspective, 
there are no pure interventions. That is, 
clinically, stimulus-based intervention 
always encompasses changes in reinforce- 
ment parameters (e.g., redesigning a 
curriculum will also produce shifts in the 
allocation of reinforcers across behav- 
iors), and reinforcement-based interven- 
tion always encompasses changes in 
stimulus parameters (e.g., a reinforcer is, 
itself, a stimulus that can be discrimina- 
tive for specific responses). These facts 
led to our decision to pool all the data for 
the two types of intervention in order to 
study how various factors modulate the 
effectiveness of PBS-based intervention 
as a whole. Because clinical practice 
always involves combining elements of 
stimulus- and reinforcement-based 

O 



intervention, we assume that pooling the 
data across the two types of intervention 
provides one plausible estimate of the 
outcomes of comprehensive positive 
behavior support applied to an indi- 
vidual. 

Influence of Demographic 
Variables 

Table 10 displays the relationship between 
different demographic variables and 
intervention effectiveness across all PBS- 
based outcomes. The first entry, for 
gender, shows that the percentage of 
success was not strongly influenced by 
gender per se; both males and females 
showed equivalent success rates (i.e., 
within a few points of 50%). 

The data on age show considerable 
variation, from a success rate of 41.1% for 
adults (i.e., participants over 20 years of 
age) to a success rate of 63.7% for 
adolescents, with preschool and school- 
age children falling between these two 
extremes. The relationship between age and 
success rate appears linear up to adoles- 
cence, and drops off sharply thereafter. 

With respect to diagnosis, the 
success rate was highest for those 
diagnosed with a combination of mental 
retardation plus autism (59.1%), and 
lowest for those diagnosed with autism 
alone (43.2%). The other two diagnostic 
categories were associated with success 
rates that fell between those just given. It 
is interesting to note that the diagnostic 
categories associated with the smallest 
number of outcomes (autism, and mental 
retardation plus autism) were also the 
two whose success rates deviated most 
sharply from the typical finding (in this 
synthesis) of a 50% success rate, suggest- 
ing the influence of sampling error rather 
than formal diagnosis per se. 
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Table 10. Relationship Between Demographic Variables 
and Outcome Effectiveness 




Number of 


Number of 


Percentage of 


Characteristic 


outcomes 


successes 


successes 


Gender 


Male 


235 


128 


54.5 


Female 


117 


60 


51.3 


Age in years 


0-4 


37 


17 


45.9 


5-12 


129 


70 


54.3 


13-19 


88 


56 


63.7 


>20 


112 


46 


41.1 


Diagnosis 


Mental retardation (MR) 


201 


106 


52.7 


Autism (Aut) 


37 


16 


43.2 


MR + Aut 


44 


26 


59.1 


MR +/or Aut plus other diagnoses 


72 


35 


48.6 


Level of retardation 


Mild 


51 


23 


45.1 


Moderate 


53 


35 


66.0 


Severe 


100 


53 


53.0 


Profound 


110 


58 


52.7 


Type of problem behavior 


Aggression 


90 


50 


55.5 


Self-injurious behavior 


132 


74 


56.1 


Property destruction 


13 


5 


38.5 


Tantrums 


13 


8 


61.6 


Single type 


248 


137 


55.2 


Combination of types 


118 


52 


44.1 



Note: Number of outcomes does not always sum to 366 due to missing data. 



Level of retardation produced a 
pattern of results similar to those just 
noted for diagnosis. Specifically, the two 
retardation levels associated with the 
greatest number of outcomes (severe and 
profound) had success rates within a few 
™;'" f s of 50%. Those associated with the 




fewest number of outcomes (mild and 
moderate) had success rates that deviated 
from the 50% rate (i.e., 45.1% and 66.0% 
respectively), again suggesting the 
influence of sampling error rather than 
level of retardation per se. 
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Table 1 1. Relationship Between Assessment Variables 
and Outcome Effectiveness 


Factor 


Number of 
outcomes 


Number of 
successes 


Percentage of 
successes 


Type of assessment 


Informal observation 


19 


8 


42.1 


Formal direct observation 


10 


6 


60.0 


Functional analysis 


105 


64 


61.0 


Combined assessments 


132 


79 


59.8 


Assessment conducted 


266 


157 


59.0 


No assessment conducted 


100 


32 


32.0 


Type of function 


Attention 


32 


20 


62.5 


Escape 


122 


75 


61.5 


Tangible 


27 


15 


55.6 


Sensory 


17 


4 


23.5 


Multiple 


52 


33 


63.5 


Assessment repeated 


Yes 


11 


4 


36.4 


No 


255 


155 


60.0 


Assessment information used 


Yes 


231 


135 


58.4 


No 


35 


22 


62.9 



Type of problem behavior produced 
success rates near 50% for those behav- 
iors associated with numerous outcomes 
(i.e., aggression and self-injurious 
behavior), and considerable deviations 
from 50% for those behaviors associated 
with few outcomes (i.e., property 
destruction and tantrums). Importantly, 
it was possible to cumulate all outcomes 
involving a single type of problem 
behavior (e.g., aggression alone, self- 
injurious behavior alone, etc.) and 
compare these to all outcomes involving 
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combinations of problem behavior (e.g., 
aggression plus self-injurious behavior, 
property destruction plus aggression, 
etc.). The success rate for outcomes based 
on single types was 55.2%, whereas the 
rate for combinations was only 44. 1%. 
Because this comparison involved large 
numbers of outcomes, the approximately 
11% spread between the two success rates 
suggests that successfully intervening on 
combinations of problem behavior may 
be somewhat more difficult than inter- 
vening on single types of problem behavior. 
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Influence of Assessment, 

Variables 

Table 11 displays the relationship between 
different assessment variables and 
intervention effectiveness. The data 
suggest that conducting an assessment 
can have a considerable impact on 
success rates. Thus, conducting some 
kind of assessment (i.e., informal 
observation, formal direct observation, 
functional analysis, or any combination of 
these three) was associated with a success 
rate of 59.0%, whereas a failure to 
conduct assessment was associated with a 
success rate of 32.0%. Functional analysis 
and combined assessment each yielded 
success rates close to 60%, as did formal 
direct observation. Informal observation 
produced the lowest success rate (42.1%); 
however, both informal observation and 
formal direct observation were based on 
very small numbers of cases, and these 
results may reflect sampling error. 

The success rates associated with 
type of function were always within a few 
points of 60% irrespective of whether the 
function was attention, escape, tangible, 
or multiple types (i.e., the problem 
behavior was maintained by more than 
one factor). The one exception to this 
finding involves the sensory function, 
which produced a dramatically lower 
success rate (23.5%). The latter success 
rate may reflect the fact that sensory 
functions are nonsocial in nature, 
whereas all the other functions are 
socially mediated. Alternatively, the 
difference may merely reflect sampling 
error given the fact that the data on 
sensory function were based on a very 
small number of outcomes. 

Table 1 1 also addresses the issue of 
whether repeating functional assessment 
over time influences success rates. The 
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data showed that repeated assessments 
were associated with lower success rates 
(36.4%) than single assessments (60.0%). 
However, the former rate is difficult to 
interpret because it is based on only 1 1 
outcomes, again plausibly reflecting 
sampling error. Alternatively, this finding 
may reflect the fact that repeated 
assessments were more likely if the initial 
intervention proved unsuccessful or if the 
factors controlling the behavior were 
more complex. 

Finally, the data show that success 
rates were equivalent and high whether or 
not the assessment data were used to 
design an intervention. However, the 
failure to use the assessment data was a 
characteristic of only 35 outcomes. In 
addition, it was not possible to determine 
from the published reports whether mere 
knowledge of assessment data nonethe- 
less influenced choice of intervention in 
subtle ways not articulated by the 
investigator. 

Systems Change 

Table 12 shows that the two systems- 
change variables may have had an effect 
on success rates. Thus, when significant 
others (e.g., teachers, parents, job 
coaches) altered their own behavior as 
part of a systematic intervention, the 
success rate was 55.2%, which was higher 
than the 41.8% success rate associated 
with interventions that did not involve 
behavior change on the part of significant 
others. Likewise, environmental reorgani- 
zation was associated with a success rate 
of 65.0%, higher than the rate obtained 
without such organization (50.9%). 
However, the latter finding must be 
interpreted cautiously, because there were 
so few outcomes associated with environ- 
mental reorganization. 

i 
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Table 12. Relationship Between Systems- 
Change Variables and Outcome Effectiveness 


Factor 


Number of 
outcomes 


Number of 
successes 


Percentage of 
successes 


Significant others change 








Yes 


268 


148 


55.2 


No 


98 


41 


41.8 


Environmental reorganization 








Yes 


20 


13 


65.0 


No 


346 


176 


50.9 



Table 13. Relationship Between Ecological Validity 
and Outcome Effectiveness 



Factor 


Number of 
outcomes 


Number of 
successes 


Percentage of 
successes 


Intervention agent 


Typical 


154 


94 


61.0 


Atypical 


196 


87 


44.3 


Intervention setting 


Typical 


126 


61 


48.4 


Atypical 


239 


128 


53.5 


Intervene in all relevant contexts 


Yes 


76 


42 


55.3 


No 


246 


127 


51.6 



Note: Data based on combinations of both typical and atypical intervention agents and 
settings were omitted, because there were too few outcomes. 



Ecological Validity 

With respect to ecological validity, the 
data (Table 13) show that the type of 
intervention agent may make a difference. 
Typical agents (e.g., parents, teachers) 
produced a success rate of 61.0%, 
whereas atypical agents (e.g., psycholo- 
gists, researchers) had a success rate of 
only 44.3%. With respect to type of 
setting, the success rates for typical (e.g., 



home, integrated school) versus atypical 
(e.g., clinic, institution) were roughly 
comparable. Comparability in success 
rates was also seen for interventions that 
took place in all relevant contexts versus 
those that did not. 

Medication 

In light of the well-developed pharmaco- 
therapy literature alluded to earlier (Reiss 
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8c Aman, 1998; Schaal 8c Hackenberg, 
1994; Schroeder 8c Tessel, 1994; Thomp- 
son et al„ 1991), it seems plausible that 
medication might be another variable 
that modulates the effectiveness of PBS. 
However, in many of the studies exam- 
ined, no reference was made to medica- 
tion, so its use is an unknown. In still 
other studies, reference was made to 
medication, but the dosage level was not 
indicated nor was the duration of drug 
use specified. A minority of studies 



0 




provided details on dosage level and 
duration but did not report data on how 
medication and PBS interrelated. Thus, 
the unique contributions made by 
medication' versus PBS are not known at 
this time. More important, perhaps, the 
possibility that combinations of PBS and 
medication produce synergistic effects 
has not been examined systematically 
and is clearly a topic that merits future 
attention from researchers. 
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DISCUSSION 



The results of our synthesis bear on 
several important issues: (a) potential 
biases in the retrieved literature, 

(b) inferences based on the results, 

(c) effectiveness of PBS, and (d) implica- 
tions for future research. 

Potential Biases in the 
Retrieved Literature 

Rigor Versus Relevance? 

Our exclusion criteria (Table 2) were 
designed to produce a database that met 
the highest standards of methodological 
rigor currently articulated in the field. 
These standards, however, reflect a strong 
bias in favor of demonstrations of 
experimental control (internal validity), 
sometimes at the expense of demonstra- 
tions of generality (external validity). 
These standards emphasize the analysis 
of cause and effect relationships at the 
level of single individuals with a view to 
understanding process variables. This 
does not mean, however, that the single- 
subject designs that predominate in the 
field are inherently unable to address the 
issue of external validity. 

It will be helpful to review, briefly, the 
difference between group methods and 
single-subject designs with respect to the 
issue just raised. Traditional group 
methods rely on sampling procedures to 
establish external validity. Greater 
generality is possible for individuals who 
are similar to research participants on 
relevant characteristics, and generally 
external validity is less reliable for 
individuals who are different from 
research participants on those character- 
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istics. In the case of single-subject 
designs, the investigator’s ability to 
generalize to other individuals is depen- 
dent on an understanding of what factors 
controlled a participant’s behavior in 
baseline (Wolery & Ezell, 1993). Once an 
investigator understands what controls 
baseline responding, then applying 
interventions to other participants whose 
behavior is controlled by variables similar 
to those identified in the baseline will 
likely produce greater external validity 
than that achievable in the absence of 
such similarity. 

Single-subject research, now almost 
four decades old, has been invaluable in 
identifying critical variables that control 
problem behavior, raising issues concern- 
ing assessment and intervention, and 
suggesting broad guidelines for 
remediation. It is also true, however, that 
this style of rigorous experimental 
research is easiest to carry out in con- 
trolled situations. This contributed to the 
large number of outcomes in our 
database that involved atypical interven- 
tion agents (especially researchers and 
other expert professionals), atypical 
settings (particularly noncommunity 
settings in which key variables could be 
easily monitored and manipulated), and 
restricted venues for intervention (i.e., a 
lack of intervention in all relevant 
contexts). Short-term process studies 
conducted in restricted contexts also tend 
to underemphasize repeated assessment, 
multicomponent intervention, and 
measures of generalization, maintenance, 
and validated lifestyle change. Yet, as we 
noted earlier, these are precisely the 
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dimensions that define relevance 
(external validity), because without them 
there is no demonstration that an 
intervention (a) is readily applicable in 
the community or (b) changes peoples’ 
lives in broad, meaningful ways. In view 
of what we have just outlined, it is not 
surprising that the journal best known for 
its emphasis on experimental analysis 
( Journal of Applied Behavior Analysis) 
accounted for almost half (42%) of the 
articles included in the database (Table 1) 
whereas the journal best known for its 
applied community orientation ( Journal 
of The Association for Persons With Severe 
Handicaps) accounted for only 9% of the 
articles included. 

Have researchers chosen rigor over 
relevance? It seems so. Is this choice the 
only one possible for the field? We 
think not. 



Rigor Over Relevance? 

With respect to the issue of rigor over 
relevance, there is much research, 
including some in our database, that does 
in fact demonstrate a systematic concern 
for external validity and all the variables 
associated with it. In addition, many of 
the excluded articles, while not meeting 
our methodological criteria, easily met 
important criteria for external validity, 
and for that reason will be highlighted 
shortly. Finally, our review of the research 
literature should not be seen as an exact 
reflection of current clinical practice. 
There is a vast nonresearch (practitioner) 
literature that reports successful, exter- 
nally valid applications of PBS (e.g., 
Copeland, 1997; Hays, 1997; Jones, 1997; 
Kincaid, 1992, 1996; Lucyshyn &Albin, 
1993; Metlen, Majure, & Stroll-Reisler, 
1996; Metz, 1992a, 1992b, 1992c; The 



Family Connection staff, DeVault, Krug, & 
Fake, 1996; Tifft, 1996; Turnbull & 

O ’ J U> 1996; Virginia Institute for 
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Developmental Disabilities, 1996). The 
challenge therefore is how best to view 
the bias toward controlled experimental 
research so that new perspectives can be 
created that build constructively on that 
bias while at the same time moving 
beyond it. 

The bias toward controlled experi- 
mental research is a legitimate beginning 
for the evaluation of the PBS approach. 
Recall that two questions critical to this 
synthesis (and discussed shortly) involve 

(a) whether PBS is effective and (b) what 
factors modulate its effectiveness. These 
questions can be definitively answered 
only through careful experimental 
analysis that includes appropriate 
controls to ensure internal validity. In the 
absence of these controls, interpretation 
of the database would be speculative at 
best. Nonetheless, the portions of the 
database that emphasize internal validity 
while minimizing the focus on external 
validity are a useful beginning for the 
field, because they increase our confi- 
dence that (a) PBS procedures do have 
demonstrable effectiveness and 

(b) certain variables are causal in 
modulating that effectiveness. 

The portions of the experimental 
database, however scant, that move 
beyond internal validity issues to 
demonstrate ecological validity show that 
it is possible to produce meaningful 
change under more naturalistic condi- 
tions. The question then becomes how 
best to extend the database under 
naturalistic conditions. For this, one must 
turn to an examination of some of the 
excluded research articles that we will 
review shortly. At this point in the 
development of the field, it is clear that 
more controlled experimental research in 
community settings is a legitimate future 
direction for the extension of PBS. 
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The nonresearch literature that 
focuses on consumer needs and interests 
helps to clarify the gap between the 
research literature (both included and 
excluded studies) and what key consum- 
ers value, thereby providing a heuristic 
for moving beyond what researchers 
presently offer. 

In sum, the biases in the database 
serve a useful function in establishing a 
trustworthy foundation which, together 
with the heuristic elements inherent in 
the excluded research articles and 
nonresearch literature, provide direction 
for the further development of the PBS 
approach. 

Is Rigor Over Relevance the 
Only Choice? 

With respect to whether one must choose 
between rigor and relevance, thoughtful 
consideration suggests that this issue 
needs to be restated. Specifically, there is 
no absolute standard for rigor. Rigor is a 
function of context. Currently the “gold 
standard” for rigor is the laboratory 
experiment, but, as we have seen, this 
standard is generally unrealistic in 
complex community settings. 

Rigor needs to be defined in terms of 
its contextual parameters. Analog 
demonstrations ought to be subjected to 
criteria for rigor that characterize 
laboratory research, in which there is an 
emphasis on highly controlled situations 
that permit cause and effect statements. 
In contrast, applied community demon- 
strations ought to be subjected to criteria 
for rigor that reflect the realities of 
carrying out research in situations that 
typically offer fewer opportunities for 
manipulating one variable at a time, that 
frequently contain multiple interacting 
variables, that require nonresearchers/ 
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nonexperts to implement assessment and 
intervention, that demand that interven- 
tion be conducted over protracted 
periods of time, and that entail altering 
social systems to produce validated 
lifestyle change. 

These applied community criteria for 
rigor have not been systematically 
articulated in the literature, and, there- 
fore, represent an evolving feature of the 
field, one that, at its heart, is tantamount 
to developing a new applied science. From 
a researcher’s perspective, this task 
represents the greatest challenge offered 
by the PBS approach. 

It is important to note that the 
laboratory-style experiment remains the 
only method for determining, definitively, 
cause and effect relationships. In this 
case, advances in basic knowledge will 
still depend on using some variant of the 
traditional experiment. However, 
multidimensional projects that focus on 
intervention efficacy need to be evaluated 
as well, and, although they may not produce 
new basic knowledge, they generate a 
type of applied knowledge that is crucial 
to advancing the practice of PBS in 
naturalistic contexts. Some of the research 
articles excluded from our database 
represent an important step in moving 
beyond the current bias favoring experi- 
mental studies, and toward incorporating 
the best information generated by those 
studies into a research style that focuses 
on issues of external validity. 

External Validity of 
Excluded Studies 

Of the 107 articles excluded for one or 
more methodological reasons, a number 
were noteworthy in that they demon- 
strated the viability of PBS in meeting 
important criteria for external validity 
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(e-g.. Cameron, Ainsleigh,& Bird, 1992; 
Dadson & Horner, 1993; Horner, Close et 
al., 1 996; Horner, Vaughn, Day, & Ard, 
1996; Lucyshyn, Olson, & Horner, 1995; 
Malette et al., 1992; Northup et al., 1994; 
Smith, 1985; Smith & Coleman, 1986). 
Although we did not examine this 
literature using the detailed analyses 
applied to the included studies, it maybe 
helpful here to review one representative 
study that reflects a concern with external 
validity criteria. 

One illustrative study by Horner, 
Close et al. (1996) involved PBS interven- 
tion for 12 adults who had been institu- 
tionalized for many years. These indi- 
viduals were identified by staff as among 
the most challenging in the entire 
institution. The approach taken involved 
a number of PBS interventions whose 
internal validity had been well estab- 
lished in numerous experimental studies 
of the type included in our database. 

Thus, the intervention adopted was built 
on a solid foundation of experimental 
research. Importantly, however, numerous 
variables associated with external validity 
concerns were combined with the core 
interventions in order to produce an 
approach that proved viable in addressing 
the wider needs of the individuals 
involved. Specifically, the entire interven- 
tion was implemented by typical inter- 
vention agents (support staff) rather than 
by experts or researchers. A supported 
living model was put in place that focused 
exclusively on home and community 
(typical settings) as the site of interven- 
tion. Evaluation occurred over a period of 
4 years (long-term maintenance). 

Because of the changing nature of the 
living situation, repeated assessments 
were routinely conducted, and interven- 
tion was carried out in all relevant 
contexts. As the natural contexts involved 
O 




multivariate control of problem behavior, 
the intervention itself was multicompo- 
nent and included reorganizing the 
environment (rearranging the physical 
setting and schedules), changing task 
features, teaching new skills, altering 
reinforcing consequences for positive 
behavior as well as applying extinction to 
problem behavior, and enhancing health 
and safety factors. The results of this 
comprehensive PBS strategy demon- 
strated decreases in major problem 
behaviors, greater physical and social 
inclusion, and stability or improvement in 
health and safety. Further, both families 
and direct support staff validated these 
changes with respect to variables that 
included community integration, social 
relationships, problem behavior, and 
overall quality of life. Thus, validated 
lifestyle change was evident. 

This study, like the others cited 
earlier, is instructive for two reasons. 

First, it demonstrates that a comprehen- 
sive PBS approach that is responsive to 
external validity concerns can yield 
improvements in problem behavior while 
producing validated lifestyle change. 
Second, it demonstrates the need to 
consider anew how we define rigorous 
applied science; if we do not, we will 
continue to underestimate the scope and 
value of PBS applications in community 
contexts. It is encouraging to note that 
emerging research addresses the issue of 
what constitutes rigorous community- 
based science (e.g., Lucyshyn, Albin, & 
Nixon, 1997). It is becoming clear that 
such research will almost certainly need 
to redefine what is acceptable in the 
domains of assessment, intervention, and 
outcomes, a point to which we shall 
return later. 

At this point, it is useful to examine 
various features of the database while 
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keeping in mind the external validity 
criteria we have just discussed. In this 
manner, gaps in the field can begin to be 
identified and then elaborated upon in 
the final section of the discussion that 
deals with the differing perspectives and 
needs of consumers (nonresearchers). 

Drawing Inferences 
from the Results 

In this section, we use the database in 
order to address the first four research 
questions posed earlier. 

How Widely Applicable 

Several researchers have raised the 
legitimate question of whether PBS might 
be limited in the scope of its applicability. 
Specifically, they have cautioned that the 
approach has, in the past, often focused 
on individuals who are relatively easy to 
deal with because they are young and 
have minor disabilities, high levels of 
cognitive functioning, and mild problem 
behavior (Axelrod, 1987; Feldman, 1990; 
Mulick & Linscheid, 1988). The present 
data (Table 4) make clear, however, that 
the field has developed to the point that 
difficult cases are now being addressed. 
Thus, (a) over half the cases in our 
database were adolescents or adults; 

(b) serious disabilities (i.e., mental 
retardation, autism) were well repre- 
sented; (c) fully two-thirds of the cases 
fell within the severe/profound level of 
retardation; and (d) the types of problem 
behavior involved were among the most 
serious ones seen clinically (aggression, 
self-injury). There is, however, one area 
where the scope of applicability of PBS is 
as yet unclear, specifically, cultural 
diversity. Very few studies mention 
minority status as a participant charac- 
0 | y^-stic. More significantly, no studies 



investigated whether cultural parameters 
influenced the design of the PBS ap- 
proach used. Yet cultural factors have 
been identified as one important aspect 
of planning interventions for people with 
disabilities (Harry, Allen, 8c McClaughlin, 
1995; Harry, Grenot-Scheyer et al„ 1995). 
The systematic exploration of cultural 
factors and PBS use is a topic that awaits 
research scrutiny. 

Another concern is that the putative 
complexity of PBS may preempt its 
applicability to natural (typical) settings 
unless there is extensive involvement of 
highly trained professionals (Paisey, 
Whitney, 8c Hislop, 1990). Again, however, 
the data (Table 4) show that almost half 
of the intervention agents were not 
experts; they were parents, teachers, job 
coaches, and the like (typical agents). 
Further, in one-third of the cases, 
intervention took place in typical settings 
(homes, integrated schools), thereby 
demonstrating that the use of PBS was 
not restricted to the kinds of specialized 
settings (e.g., clinics, segregated schools) 
normally associated with expert profes- 
sionals. 

In What Ways Is the 
Field Evolving ? 

General Status of PBS 

Historically, the field of developmental 
disabilities has shown a tendency to 
embrace new intervention approaches 
with great enthusiasm and then abandon 
them with equal enthusiasm as the initial 
promise of the interventions failed to 
survive scientific scrutiny (Arendt, 
MacLean, 8c Baumeister, 1988; Green 8c 
Shane, 1994; Willemsen-Swinkels, 
Buitelaar, Nijhof, 8c van Engeland, 1995). 
Is PBS yet another passing fad? The data 
(Figure 1) suggest that this approach is 
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here to stay. Over the 12-year period of 
the review, there was a clear increase in 
the number of articles published as well 
as the number of participants and 
outcomes involved. This increase is 
especially noteworthy in view of periodic 
criticisms of PBS alluded to earlier in this 
review (e.g., see Repp & Singh, 1 990). 

Demographics 

PBS procedures have remained widely 
applicable over time to individuals with 
significant problem behavior (Figures 2- 
6). During the 12-year period reviewed, 
very young children (preschool age) 
made up only a small percentage of the 
cases; most individuals who participated 
in PBS interventions were older 
(elementary-school age to adult) and, 
therefore, presumably more difficult to 
deal with. Likewise, people diagnosed as 
having mental retardation or autism, and 
those with cognitive functioning in the 
profound or severe range were present in 
substantia] and stable numbers over time, 
again attesting to the continued applica- 
bility of PBS to individuals who are 
generally viewed as providing serious 
challenges. Importantiy, while the 
proportion of cases involving aggressive 
behavior remained stable, those involving 
dangerous self-injurious behavior and 
multiple problem behaviors (combina- 
tions) actually increased over time. In 
sum, then, PBS procedures continued to 
be applied to populations of people who 
are generally viewed as posing the 
greatest challenges. 

As noted previously, there has been 
some concern in the field that PBS 
procedures are being implemented 
primarily by highly trained experts 
(atypical intervention agents) in highly 
specialized (atypical) settings (Paisey et 
al., 1990; Scotti et al„ 1996). Our database 
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(Figure 7) demonstrated no clear trend 
with respect to type of intervention agent. 
Interestingly though, in the last time 
block (1993-1996), there were equal 
numbers of typical and atypical interven- 
tion agents, a fact that showed the 
continued heavy involvement of typical 
intervention agents at a time when the 
number of PBS studies rose dramatically 
Just as important, there appears to be a 
clear trend in the field toward applying 
PBS more often in typical settings. Thus, 
the ratio of typical to atypical settings 
has, over time, been approaching parity 
(Figure 8). The field appears to be moving 
tentatively toward greater use of PBS in 
natural contexts. 

Assessment 

For many years, investigators have 
stressed the importance of conducting an 
assessment prior to designing an 
intervention, but they have also noted the 
relative lack of such assessment in the 
published literature (Carr et al., 1990; 
Scotti et al., 1991). More recently, reviews 
of the entire spectrum of interventions 
for problem behavior have detected a 
clear movement toward greater use of 
assessment methods (Scotti et al., 1996; 
Didden et al., 1997). Our database 
confirms that these general trends in the 
field are seen also in the PBS literature. 
There has been a dramatic increase over 
time in the proportion of outcomes 
associated with prior assessment (Figure 
9). To extend previous reviews, we broke 
down the information obtained so that we 
could examine trends with respect to 
specific types of assessment (Figure 10). 

We found large increases in the use of 
functional analysis and combinations of 
assessment procedures. The sole use of 
indirect or direct observation remained a 
rarity. Our data also showed that func- 
tional analysis was most likely to be 
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associated with atypical intervention 
agents, atypical settings, and a lack of 
intervention in all relevant contexts, all 
indices of lower ecological validity. In 
contrast, informal and/or direct observa- 
tion were generally associated with 
higher levels of ecological validity. 

These data suggest that functional 
analysis is unlikely to be viewed as a 
feasible method of assessment by 
practitioners operating in naturalistic 
contexts. One recent survey (Desrochers, 
Hile, & WilliamSrMoseley, 1997) supports 
this notion. Specifically, 300 practitioners 
were asked to rank the degree of useful- 
ness of a number of assessment proce- 
dures. ABC (antecedent-behavior- 
consequences) analysis (i.e., a direct 
observation procedure) and interview 
with staff/relatives (i.e., an indirect 
observation procedure) ranked first and 
second respectively. Functional analysis 
ranked a very distant fifth in usefulness. 
Practitioners felt that a lack of environ- 
mental control, insufficient time, diffi- 
culty collecting data, lack of expertise, 
and environmental complexity all made 
functional analysis an impractical and, 
therefore, seldom used method in real-life 
settings. Taken as a whole, these consider- 
ations argue for the development and 
refinement of a new generation of 
nonexperimental assessment procedures 
that are user-friendly, practical, and 
feasible, while retaining functional 
analysis primarily as a tool for research- 
ers and occasionally to be used by experts 
for cases in which less formal methods have 
failed. There is, of course, a related strong 
need for personnel preparation programs 
and associated texts (e.g.; Snell, 1993) that 
can assist future professionals in con- 
ducting valid and practical assessments. 



Our review of the PBS literature is 
also in afccord with analyses of motivation 
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documenting that escape is the single 
most commonly identified function for 
problem behavior (Scotti et al„ 1996). In 
addition, our database shows that this 
feature is stable in that escape was 
associated with the greatest number of 
outcomes in each successive time block 
(Figure 11). These data may reflect the 
decreasing emphasis in the field on 
custodial care and a greater emphasis on 
education, involvement in meaningful 
activities, and work (Emerson et al., 

1994). Because escape-motivated 
problem behavior typically occurs in 
response to demands (e.g., academic 
tasks, home chores, work activities), the 
shift from low-demand custodial 
situations to higher-demand school and 
community-based situations may have 
provided many more opportunities for 
escape- motivated problem behavior to 
manifest itself. 

Interestingly, the proportion of 
outcomes involving multiple motivations 
is accelerating over time. The greater 
involvement of people with disabilities in 
typical settings, noted earlier, may be 
related to the increase in outcomes 
associated with multiple motivation; the 
complexity of school, work, and home 
situations (typesettings) compared to 
the more restricted, less complex atypical 
settings may provide increased opportu- 
nities for multiple factors (e.g., escape, 
attention, tangibles) to exert their 
influence on problem behavior. 

The data on whether assessment was 
repeated provide some important insights 
on how the field is progressing. Thus, 
although repeated assessment was 
associated with only 11 outcomes, 10 of 
these occurred in the most recent time 
block. Significantly, repeated assessment 
was most likely to be carried out in 
typical settings and by typical interven- 



76 



63 



Chapter 4 



tion agents. In contrast, a lack of repeated 
assessment was most likely to be associ- 
ated with atypical agents and atypical 
settings. These findings make sense given 
recommendations that assessment should 
be repeated whenever there are important 
changes in an individual’s life situation 
(Carr et al„ 1994; O’Neill et al„ 1997a & 
b). In typical settings (e.g., home, school), 
ongoing change is the norm; in atypical 
settings, ongoing change may be less of a 
factor, because such settings are often 
more restricted in terms of activity 
schedules, social relationships, and 
reinforcer accessibility. Although the 2:1 
ratio of atypical to typical settings seen in 
the database (Table 4) is consistent with 
the lopsided ratio of nonrepeated 
assessment to repeated assessment, the 
extremely low frequency of the latter does 
nonetheless represent a relative short- 
coming of the field with respect to 
embracing one standard of best practice. 

Intervention 

The data (Figure 12) show that both 
stimulus-based and reinforcement-based 
interventions have remained an enduring 
feature of the field. However, there has 
been an important change toward a 
predominance of stimulus-based 
procedures. This change likely reflects a 
surge of interest, more generally in the 
field, in making education a priority for 
people with disabilities. Educationally 
relevant and popular procedures, such as 
interspersal training, choice, curricular 
modification, errorless learning, and 
prompting, are all stimulus-based and 
integral to the promotion of learning in 
academic and related contexts (Luiselli & 
Cameron, 1998). 

As noted earlier, the PBS philosophy 
does not view people with disabilities as 
being passive recipients of intervention 
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for problem behavior. Instead, they are 
seen as participating in systems that 
precipitate problem behaviors, and such 
systems are in need of change. 

The data on systems change (Figure 
13) suggest a dramatic increase over time 
in the requirement for behavior change 
on the part of significant others. 
Caregivers and support persons are most 
recently (1993-1996) frequently required 
to change aspects of their own behavior 
as one component of intervention. This 
finding reflects the reciprocity of 
behavior change that is one hallmark of 
PBS intervention. 

In contrast, the data on environmen- 
tal reorganization, an additional aspect of 
systems change, fail to show an increasing 
trend but remain at low levels. This 
finding may reflect the abiding predomi- 
nance of atypical settings over typical 
settings (Figure 8) as the site for interven- 
tion. In typical settings, a major priority 
is commonly given to selecting engaging 
activities, enriching lifestyle through 
community integration, providing choice, 
systematically providing respite services, 
and offering other opportunities related 
to broad environmental reorganization/ 
restructuring (Emerson et al„ 1994). 
Atypical settings are very often more 
restrictive and institutional, limiting 
opportunities for such types of broad 
change. Thus, the imbalance that favors 
the use of atypical settings over typical 
settings likely contributed greatly to the 
lack of focus on environmental reorgani- 
zation characterizing the current data- 
base. The accelerating interest in using 
typical settings as well as the current 
paucity of data on environmental 
reorganization should, together, prompt 
the field to explore and analyze this 
aspect of systems change further as one 
more element of best practice. 
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Some of what has just been dis- 
cussed also bears on the finding that the 
application of intervention in all relevant 
contexts was uncommon (Figure 14). 
Further analyses demonstrated that 
intervention in all relevant contexts was 
most likely to be associated with typical 
agents and settings, and least likely with 
atypical agents and settings. Other data 
showed that intervention in all relevant 
contexts was closely associated with 
combined interventions (i.e., stimulus- 
based plus reinforcement-based), 
whereas its absence was closely associ- 
ated with the use of single interventions 
(i.e., either stimulus-based or reinforce- 
ment-based). One explanation is that 
atypical agents and settings are often 
used in analog (laboratory-style) research 
designed to study basic principles rather 
than to demonstrate broad clinical 
changes; so the scope of such interven- 
tions may be quite limited (i.e., not all 
relevant contexts are involved and single 
interventions are common). In contrast, 
with typical agents and settings, the focus 
is often on demonstrating and evaluating 
the use of interventions across all 
pertinent aspects of an individual’s life, so 
such interventions may more likely 
involve all relevant contexts and the 
application of combined interventions. 
The paucity of data on intervention in all 
relevant contexts may reflect a strong 
interest, at least in the literature sample 
reviewed, in elucidating basic principles 
at the expense of demonstrating broad- 
spectrum behavior change. Because the 
latter activity is at least as important as 
the former, one priority for the field 
would be to redress this imbalance. 

The pattern we have been describing, 
involving typical versus atypical agents 
and settings, is seen once again in the 
analysis of combined interventions (i.e., 
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those involving both stimulus-based and 
reinforcement-based as part of the same 
intervention package). Figure 12 shows 
no trend for the increased or decreased 
use of combined interventions. However, 
additional analyses make clear that 
combined interventions, when they did 
occur, were more likely to be conducted 
by typical intervention agents and in 
typical settings. Noncombined interven- 
tions were more likely to be conducted by 
atypical agents and in atypical settings. 
This pattern also conforms to the 
distinction between analog research and 
applied research in the community. In the 
former type of laboratory-style research, 
one typically avoids combining multiple 
variables, favoring instead the manipula- 
tion of as few variables as possible to 
demonstrate cause-and-effect relation- 
ships unambiguously. In contrast, the 
latter type of research is more concerned 
with demonstrating the efficacy of 
intervention packages (i.e., the most 
potent combination of interventions). 

In sum, then, the data in Figures 1 2- 
14 yield a fairly consistent picture in terms 
of the very different research and clinical 
agenda represented by typical agents 
working in typical settings versus atypical 
agents working in atypical settings. 

A final aspect of the intervention 
data pertains to the use of non-PBS 
procedures (Figure 12). These procedures 
were much less frequently used than PBS 
throughout the entire time period 
reviewed. This finding implies that, for a 
clear majority of outcomes, PBS by itself 
was viewed by investigators as an adequate 
approach for dealing with serious problem 
behavior; the addition of non-PBS 
procedures was not seen as a necessity. 
Subsequent analyses, discussed shortly, 
corroborate and amplify this conclusion. 
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Outcomes 

For many years, PBS advocates, among 
others, have argued strongly that reduc- 
tion in problem behavior is an insuffi- 
cient outcome unless such reduction is 
associated with broader changes that 
include stimulus and response generali- 
zation, maintenance, and improvements 
in lifestyle (Carr et al„ 1994; Horner, 
Dunlap, & Koegel, 1988; Koegel et al., 
1996; Meyer & Evans, 1989; Schalock, 
1990; Turnbull & Turnbull, 1996). Our 
database (Figure 15) suggests that this 
call for best practices has gone largely 
unheeded. Relatively few outcomes 
involved demonstrations of broad change. 
Thus, no trends were seen for stimulus 
generalization and only a small increase 
was seen for response generalization. 
Demonstrations of short-term mainte- 
nance (less than 6 months) became more 
frequent over the period reviewed; 
however, demonstrations of longer-term 
maintenance dropped sharply in fre- 
quency as the duration of follow-up was 
extended. Significandy, no outcomes were 
tracked for more than 2 years, thus 
leaving unanswered the question of 
whether PBS can produce permanent 
change. Of most concern, however, is the 
finding that a focus on lifestyle change 
was uncommon, with no compelling 
upward trend. Studies meeting our 
rigorous inclusion criteria appeared more 
concerned with demonstrating initial 
efficacy and with understanding of 
process, and less concerned with more 
generalized change including improve- 
ments in lifestyle. This gap in the 
literature helps define a critical research 
priority for the future. 

The lack of focus on broad change is 
paralleled by a lack of focus on social 



validity (Figure 16). Social validity data 
were available for only a small number of 
outcomes. Modest increases over time 
were seen for questions dealing with the 
feasibility of intervention and the 
acceptability of intervention outcomes, 
but no trends were seen for questions 
pertaining to the desirability of specific 
intervention procedures and the effective- 
ness of intervention in producing lifestyle 
change. The paucity of data on social 
validation may be linked to the paucity of 
data on broad change. That is, to the 
extent that investigators have been largely 
concerned with demonstrating interven- 
tion effects in somewhat circumscribed 
contexts, it makes little sense to ask 
social validity questions that are intended 
to tap consumer satisfaction across a 
broad range of circumstances. A greater 
future emphasis on studies of broad 
behavior change should lead as well to a 
greater focus on whether such change is 
socially valid. 

How Effective Is PBS ? 

Changes in Positive Behavior 

Our database (Figure 17) documented, 
true to the definition of PBS, increases in 
positive behavior. Data available for 
slightly less than half the outcomes 
demonstrated modest to substantial 
increases in socially desirable responses 
(positive behaviors) as a consequence of 
intervention. These data are important 
because they show that the field has 
moved beyond an exclusive concern with 
reducing problem behavior per se and has 
become more committed to enhancing 
constructive aspects of individuals’ 
repertoires as a general strategy for 
dealing with problem behavior. 
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Distribution of Outcome 
Effectiveness: What Constitutes 
a Success? 

There were many instances in which PBS 
was an effective strategy for reducing the 
level of serious problem behavior (Table 
5). The two generic variations of PBS, 
stimulus-based and reinforcement -based, 
were about equally effective in producing 
successful outcomes (Tables 5 & 6). It is 
important to understand, however, that 
the field has not yet reached a consensus 
on what constitutes a success. As can be 
noted in the distribution of outcome 
effectiveness (Table 5), the degree of 
success depends on where one sets the 
cutoff criterion. Across all PBS-based 
outcomes, a criterion of 100% reduction 
(total elimination of problem behavior) 
produced a success rate of 26.8%, one 
quarter of all the outcomes. Few investi- 
gators set so demanding a criterion. 
Instead, in the present paper, as well as in 
the earlier review the senior author 
conducted for the National Institutes of 
Health (Carr et al., 1990), a 90% or more 
reduction criterion was established that 
yielded a success rate of 51.6%, one half 
of all the outcomes. Other investigators 
(Mulick & Kedesdy, 1988) have used an 
80% or more reduction criterion, which 
in the present case would produce a 
success rate of 68.0%, two thirds of all the 
outcomes. An 80% reduction criterion 
might, at first glance, appear unaccept- 
ably low. Yet clinically, if an individual 
who has been biting him- or herself an 
average of 100 times a day for many 
months were to decrease biting abruptly 
to 20 times a day, caregivers might rate 
the outcome as a qualified success. In 
contrast, even a 90% reduction in a less 
challenging self-injurious behavior (e.g., 
trichotillomania: hair pulling) might be 
\jj^ed as unimpressive and relatively 




unsuccessful. The point is that other 
variables besides percentage reduction 
contribute to the definition of interven- 
tion success. In this vein, the many 
demonstrations in the literature of 
dramatic decreases in problem behavior 
during 10-minute analog sessions 
conducted by researchers are not, a priori, 
superior to more modest decreases 
reported for interventions applied 
throughout the day in natural settings by 
typical caregivers, a serious concern that 
needs more systematic attention in the 
future. 

Table 5 also shows that for 33 
outcomes (9% of the total), problem 
behavior decreased minimally (only 0- 
19% from baseline) or increased. Our 
previous review (Carr et al., 1990) 
likewise noted negative outcomes in a 
small minority of cases. Such findings 
serve to underscore the point that no 
intervention, including PBS, should be 
assumed to be universally effective. As 
noted later, negative outcomes may reflect 
the influence of variables (e.g., non- 
optimal social systems, physiological 
factors) that attenuate the impact of PBS 
strategies. Identifying such variables 
represents yet another research priority 
for the field. 

Success Rates Pooled 
Across Outcomes 

With all the caveats just discussed in 
mind, we used a 90% reduction criterion 
as our definition of success. Using this 
criterion, it was clear (Table 6) that the 
success rate was invariant and not 
dependant on the type of intervention 
employed. Generally, about half the 
outcomes could be categorized as 
successes, irrespective of whether they 
were associated with stimulus-based or 
reinforcement-based intervention and/or 
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different combinations of these with non- 
PBS procedures. However, further 
analyses of these data highlight some of 
the complexities that are not initially 
apparent from examination of Table 6 
alone. 

Single Versus Multiple Interventions 
The fact that single interventions had a 
success rate of 51.5%, and combination 
interventions had a success rate of 52.4%, 
might make it appear as if not much is 
gained by combining stimulus-based and 
reinforcement-based intervention into a 
comprehensive approach. However, as 
noted earlier, single interventions were 
most likely to be carried out by atypical 
agents (e.g., research psychologists) in 
atypical settings (e.g., institutions) 
whereas combined interventions were 
most likely to be carried out by typical 
agents (e.g., parents) in typical settings 
(e.g., homes). The degree of control 
available in typical settings involving 
typical agents is, arguably, substantially 
less than that achievable by experts 
(atypical agents) operating in more 
restricted (atypical) settings. Therefore, 
one might expect poorer outcomes 
associated with the former as compared 
to the latter. Because this result was not 
the case (i.e., the success rates were about 
the same), one could conclude that 
combined intervention might be particu- 
larly useful in dealing with situations in 
which natural (typical) intervention 
agents must address the day-to-day 
(typical) settings in which many people 
with disabilities live. 

The Role of Consequences 
Whereas PBS interventions are proactive 
(i.e., take place in the absence of problem 
behavior and act to prevent future 
occurrences of such behavior), non-PBS 
interventions are reactive (i.e., take place 
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at the moment problem behavior is 
occurring and act as direct consequences 
for such behavior). The data (Table 6) 
suggest that non-PBS interventions that 
stress consequences may not always be 
necessary for dealing with problem 
behavior because (a) for almost three 
quarters of the database (72.1% of the 
outcomes), PBS procedures were used 
alone and (b) the rate of success was 
virtually the same with or without non- 
PBS (i.e., 5 1.5% successes with PBS alone, 
and 52.0% with non-PBS). Again, 
however, consideration of a number of 
factors suggests that this conclusion 
needs to be tempered and qualified. Here 
we will discuss two key factors, the role of 
punishment and extinction as response 
consequences. 

Punishment: A Form of Crisis 
Management 

Most practitioners and caregivers are 
concerned about what they can do at the 
moment of crisis, when dangerous 
behavior is occurring, to mitigate its 
effects. Our database (Table 7) shows that 
approximately half of the non-PBS 
interventions took the form of punish- 
ment, albeit, with one exception (water 
mist), punishments that are usually 
regarded as relatively nonintrusive. The 
fact that generally innocuous non-PBS 
procedures (added to the PBS procedures 
already in place) were sufficient without 
the addition of more highly intrusive 
procedures should be encouraging to 
those opposed to the use of the latter 
more controversial measures. As has been 
pointed out elsewhere (Carr et al., 1994), 
intervention agents frequently feel the 
need for crisis management procedures 
even when committed to a PBS approach. 
The perceived need for these procedures 
cannot easily be dismissed. The impor- 
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tant message from our database, however, 
is that when additional non-PBS proce- 
dures seemed necessary, they were 
relatively benign, and relatively rare. 

Extinction: A Crucial Consequence ? 

Our database (Table 7) also showed that 
approximately half of the non-PBS 
interventions took the form of extinction. 
There are conceptual grounds for 
assuming that extinction may not always 
be necessary, but there are additional 
grounds that support its use. Conceptu- 
ally, both the Matching Law and the 
notion of response efficiency may 
plausibly obviate the use of extinction. 

The Matching Law states that the relative 
rate of a response is a function of the 
relative rate of reinforcement for that 
response (McDowell, 1982). Thus, if an 
individual receives all his or her rein- 
forcement contingent on aggressive 
behavior, that behavior would occur at a 
high frequency. But suppose, through PBS 
intervention, the individual acquired 
many new skills (responses) that also 
provided access to reinforcement. Now 
this person would not be solely depen- 
dent on aggression as a means for 
accessing reinforcers. Thus, even if 
aggressive behavior were reinforced at the 
same level as it had been prior to PBS 
intervention, the relative rate of this 
behavior would now be much lower 
because of the presence of many new 
skills (responses) that generate additional 
reinforcement. So the absence of an 
extinction contingency for aggressive 
behavior would not necessarily matter, 
because many available competing 
positive responses reduced the relative 
rate of reinforcement for aggression with 
the result that aggression would now be 
less frequent. 

Response efficiency is another factor. 
0 i show that if two responses both 




access the same reinforcer, the more 
efficient of the two is likely to predomi- 
nate (Horner & Day, 1991). Let us sayan 
individual acquired a new response (e.g., 
communication) as a result of PBS 
intervention. But then the individual 
continued to receive reinforcement for 
aggressive behavior (i.e., no extinction 
contingency). If the new response was 
more efficient than the aggressive 
behavior, the new response would 
predominate, and aggression would 
become less frequent. 

The possible role of the Matching 
Law and response efficiency in decreasing 
incidents of problem behavior in the 
absence of extinction may explain why so 
few outcomes in the database were 
associated with the use of extinction. 
Investigators may simply have found it to 
be unnecessary. Nonetheless, a variety of 
considerations support its use. First, 
many investigators, while not providing 
formal data on the use of extinction, 
made passing references to “ignoring” 
problem behavior or “not rewarding” it, 
both statements implying that some 
degree of extinction was in effect. Thus, 
the number of outcomes associated with 
extinction (Table 7) may represent an 
underestimate. Second, and more 
critically, the Matching Law itself suggests 
why extinction may be important. 
Specifically, if in addition to strengthen- 
ing new skills that access the reinforcers 
previously associated with problem 
behavior alone, one were also to discon- 
tinue the reinforcement of the problem 
behavior itself (extinction), then one 
would be greatly increasing the relative 
rate of reinforcement for the new 
(positive) skills. This strategy would also 
have the effect of increasing the relative 
frequency of positive behavior. In this 
crucial respect, extinction could, on 



82 



69 



Chapter 4 



conceptual grounds, be viewed as an 
important procedure because it directly 
potentiates the display of positive 
behavior. On these grounds alone, it 
would be prudent to include extinction as 
part of any PBS-based approach. 

Stimulus Generalization, 
Response Generalization, 
and Maintenance 

These three measures of generalization 
were typically associated with small 
numbers of outcomes. Yet, with a 90% 
reduction criterion for problem behavior, 
about two-thirds of the outcomes were 
successes for stimulus generalization and 
for the various durations of maintenance. 
Response generalization was much less 
successful, with small gains in appropri- 
ate behavior characteristic of most of the 
outcomes. The scant attention paid to 
these three measures of generalization 
was criticized in our earlier review of PBS 
intervention (Carr et al„ 1990). Unfortu- 
nately, this gap in best practices remains 
an enduring feature of the field; there 
remains a strong bias toward demonstrat- 
ing experimental control of problem 
behavior in restricted situations rather 
than investigating more widespread 
change. 

Full implementation of the PBS 
approach would significantly increase the 
database pertinent to generalization 
measures. Specifically, a greater focus on 
assessing and remediating problem 
behavior in all relevant contexts (not just 
analog settings) would, of necessity, teach 
specific alternatives to problem behavior 
in many different situations (multiple 
exemplar training), a factor known to 
promote stimulus generalization (Stokes 
& Baer, 1977). Likewise, extensive 
assessment and intervention for deficient 



behavior repertoires would teach and 
strengthen a wide variety of responses 
(communication, self-management, 
social skills) whose increase might well 
change many other behaviors in a 
response generalization paradigm (Carr, 
1988). Finally, because optimum PBS 
intervention is not saltatory but, rather, 
continuous, ongoing, and permanent, 
maintenance is potentially a natural 
consequence of the approach. To state the 
matter differently, the hallmark of the 
PBS philosophy is to remediate deficien- 
cies in all contexts (i.e., deficient environ- 
ments and deficient skill repertoires). 
Once this remediation has occurred, 
agents take steps to ensure that the 
contexts do not deteriorate to their initial 
levels of deficiency. In this manner, 
maintenance is promoted indefinitely. 

Lifestyle Change and 
Social Validity 

The ultimate goals of PBS intervention 
are to enable individuals to live more 
normalized lives (lifestyle change) and to 
have key consumers (e.g., caregivers, 
support persons, and, if possible, the 
people with disabilities themselves) agree 
that the intervention and its effects are 
worthwhile (social validity). A strong 
movement developing in the field 
emphasizes the centrality of these goals 
in service provision and remediation 
efforts (Dennis, Williams, Giangreco, 8c 
Cloninger, 1993; Hughes, Hwang, Kim, 
Eisenman, 8c Killian, 1995; Risley, 1996; 
Sands, Kozleski, 8c Goodwin, 1991; 
Schalock, 1990, 1996; Turnbull 8c 
Turnbull, 1996). Yet the database does not 
reflect this emphasis. Specifically, only a 
small number of outcomes focused on 
these issues. Data on successful lifestyle 
change were taken on only 8 out of the 
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230 participants in our sample, and these 
successful outcomes were partially 
validated by a likewise small sample of 
consumers, none of whom was an actual 
participant. Clearly, a large gap exists 
between the stated goals for PBS inter- 
vention and the intensive and systematic 
empirical exploration of these goals. 

Several factors have hindered the 
development of a larger database. First, as 
noted earlier, our rigorous inclusion 
criteria resulted in the elimination of a 
number of studies (on methodological 
grounds) that did demonstrate validated 
lifestyle change. The interpretability of 
these studies was problematic, however, 
primarily because the standard of rigor 
that exists in many parts of the field 
emphasizes a degree of experimental 
control that is most easily achieved in 
analog situations. Perhaps that is why so 
many outcomes were associated with 
restricted (atypical) settings in which the 
intervention was carried out by expert 
(atypical) intervention agents in re- 
stricted circumstances (i.e.,not all 
relevant contexts). Tightly controlling the 
context in this manner greatly increases 
the prospects of achieving internal 
validity but does so at the expense of 
developing a truly applied technology 
that has the multidimensional properties 
needed for addressing goals larger than 
experimental control, namely, validated 
lifestyle change. As noted previously, rigor 
is not an absolute construct but, rather, a 
function of context. Thus, the standards 
defining rigor for analog studies that 
stress the analysis of process will almost 
certainly have to be different from the 
standards defining rigor for naturalistic, 
community-based studies that stress the 
analysis of outcome. 
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What Factors Modulate the 
Effectiveness of PBS? 

Demographic Variables 

Although success rates varied within 
many of the demographic variables (e.g., 
on the age variable, success rates were 
considerably higher for adolescents than 
for adults), comparison of the present 
database (Table 10) with that obtained in 
previous related reviews (e.g., Carr et al., 
1990; Scotti et al., 1991) does not reveal 
any replicated trends. One could not 
conclude, for instance, that problem 
behavior in adults is consistently more 
difficult to remediate than problem 
behavior in adolescents. In addition, the 
literature contains no conceptual basis for 
inferring that variations in demographic 
variables, such as gender, age, diagnosis, 
retardation level, or topography (type) of 
problem behavior, ought to bear a 
systematic relationship to intervention 
success. 

We intentionally included an 
additional variable not explored in 
previous reviews: the relative difficulty in 
dealing with single types of problem 
behavior versus combinations of types. 
Data indicated that the latter was 
associated with lower success rates; this 
may be due in part to the increased 
challenge of intervening on multiple 
types of problem behavior. It may be that 
multiple problem behaviors serve 
multiple functions, thereby requiring the 
design of interventions that are more 
complex and, therefore, more prone to 
failure. Alternatively, the great effort 
required to deal with diverse problem 
behavior may render support people less 
likely to persevere in intervention efforts, 
resulting in poor outcomes. In any case, 
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the analysis of single versus combination 
problem behavior has not been ad- 
equately explored in the literature; the 
possibility that this factor may modulate 
intervention effectiveness makes it 
worthy of study. 

Influence of Assessment 
Variables 

One of the most striking findings was 
that the success rate for interventions 
based on some form of functional 
assessment was almost twice that of 
interventions not based on a prior 
assessment (Table 1 1). This finding 
appears robust, having been noted also in 
several previous reviews (Carr et al., 1990; 
Didden et ah, 1 997; Scotti et ah, 1 99 1 ). 

The relationship between assessment and 
successful outcomes suggests that the 
field needs to develop assessment 
technology further; as noted, the pre- 
dominant tool is currently functional 
analysis, and this type of assessment 
requires a level of expertise and a degree 
of experimental control that is frequently 
absent in community-based settings 
involving typical intervention agents. We 
return to this point when we consider 
what gaps in knowledge exist in the field 
and how best to address them. 

With respect to type of function, it 
was clear that outcome success was 
uniformly substantial (i.e., an approxi- 
mately 60% success rate) across the 
various functions with one notable 
exception: The sensory function was 
associated with an extremely low success 
rate. Although this finding might simply 
reflect sampling error based on the small 
number of outcomes associated with the 
sensory function, there is a noteworthy 
alternative explanation. Specifically, all 
the other functions were socially medi- 
ated; the sensory function was not. One 
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interpretation of this finding is that PBS 
intervention is not pertinent to problem 
behavior that lacks social mediation. A 
more optimistic interpretation is that the 
field has not as yet adequately addressed 
nonsocially mediated problem behavior 
within a PBS framework; if it were to do 
so, success rates would rise. Certainly, 
there are instances of problem behavior 
related to physiological variables that 
would logically appear to be outside the 
purview of a PBS approach (Gardner & 
Whalen, 1996; Guess &Siegel-Causey, 
1995; Lowry &Sovner, 1992; Schroeder & 
Tessel, 1994). For example, it has been 
known for some time that certain genetic 
conditions such as Lesch-Nyhan syn- 
drome may cause self-injurious behavior 
(Seegmiller, 1972). Whether sensory- 
based behavior is likewise outside the 
PBS approach is currently an empirical 
question. One working hypothesis 
postulates that extending PBS interven- 
tions so as to provide nonproblem- 
behavior alternatives for accessing 
relevant sensory stimuli might under- 
mine sensory-based problem behavior, 
thereby improving success rates (Favell et 
al., 1982; Prosser, 1988; Smith, 1986). 

Finally, the small sample of outcomes 
associated with repeated assessment is 
not in keeping with best practices. If PBS 
is truly an approach designed to deal with 
problem behavior across all relevant 
contexts for protracted periods of time, it 
strains credibility to believe that a single 
assessment would suffice to understand 
the totality of problem behavior across 
changing circumstances over many years. 
One interpretation of the relatively low 
success rate associated with repeated 
assessment is that the field has not yet 
explored this issue with the same 
intensity and detail as it has explored the 
issue of short-term analog assessment in 
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restricted settings. Developing a larger 
database on this topic could help identify 
and refine the factors critical for ensuring 
meaningful repeated assessment. 

Systems Change 

Recall that a critical aspect of PBS 
intervention is the remediation of 
deficient contexts in an effort to normal- 
ize them. This requires change in systems 
and not just in the person with disabili- 
ties. The data (Table 12) show that change 
in the behavior of significant others was 
frequent and associated with success 
rates higher than when the significant 
others were not required to alter their 
behavior. One mechanism related to this 
greater success may involve the degree of 
support the significant others provide for 
newly enhanced skills displayed by the 
person with disabilities. Thus, if the 
person with disabilities begins to use 
communication rather than problem 
behavior as a way of accessing attention, 
then greater responsivity to bids for 
attention ensures that the new communi- 
cative behavior will be strengthened and 
better able to compete with problem 
behavior. Likewise, being able to “read” 
when a person with disabilities is seeking 
attention nonverbally (e.g., via furtive 
glances toward the significant other) and 
then responding to these nonverbal cues 
prior to the display of problem behavior 
represents another change in behavior 
that serves, in this instance, to prevent 
further display of problem behavior. The 
type and magnitude of behavior change 
shown by significant others as well as 
their relation to intervention success have 
been studied only minimally in the 
literature. Our database implies that it 
would be fruitful to explore such change 
further by focusing on issues of reciproc- 
ity (i.e.,both parties in a social exchange 
O 




alter their behavior) rather than the more 
traditional emphasis on unilateral 
strategies (i.e., only the person with the 
problem behavior is the focus of interven- 
tion, and the primary goal is for that 
person to change). 

The data on environmental reorgani- 
zation, though scant, suggest that this 
molar approach can produce higher 
success rates than those approaches from 
which it is excluded or minimized. Here 
the effective mechanism may relate to the 
fact that broad environmental change 
(e.g., altering personnel, reorganizing 
activity schedules, changing the physical 
properties of the home and/or school) 
often provides an array of discriminative 
stimuli and setting events that potentiate 
the display of positive behaviors (Carr, 
Carlson, Langdon, Magito-McLaughlin, & 
Yarbrough, 1998) that, in turn, compete 
with the problem behavior. Importantly, 
our earlier discussion noted that our 
database includes a preponderance of 
atypical settings involving atypical 
intervention agents who fail to address 
problem behavior in all relevant contexts. 
This fact may account for the small 
database related to environmental 
reorganization, because any approach 
that minimizes naturalistic variables 
simultaneously limits the scope of 
opportunities available for instituting 
molar systems change. The solution to 
this dilemma lies in new research 
priorities that stress the inclusion of 
naturalistic variables the presence of 
which would motivate researchers to 
analyze and evaluate broad environmen- 
tal change. The conceptual underpinnings 
for this expanded approach are clearly 
evident in the literature on ecological 
systems theory (Bronfenbrenner, 1989). 
This theory is compatible with the 
operant perspective but also suggests 
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additional useful practices for under- 
standing the individuals behavior in 
context and for planning interventions 
oriented toward broad systems change. 
The developmental literature provides 
evidence that for parents the ongoing 
presence of informal social support that 
includes (a) multiple, high-quality, stable 
relationships and (b) assistance from 
members of a broader interpersonal 
network correlates strongly with parents’ 
viewing their child’s behavior as less 
troublesome or difficult (Dunst,Trivette, 
& Jodry, 1997). Observations such as 
these should spur the field into examin- 
ing problem behavior from the perspec- 
tive of broader ecological variables. 

Ecological Validity 

One might anticipate that the higher 
degree of experimental control that 
frequently characterizes interventions in 
restrictive situations (i.e., those involving 
atypical settings and interventions that 
do not occur in all relevant contexts) 
might produce higher success rates, 
particularly when the intervention agent 
is an expert/professional. The database 
(Table 13) does not confirm this expecta- 
tion. Instead, typical agents are associated 
with higher success rates. Further, the 
success rates involving typical versus 
atypical settings, and the presence versus 
absence of intervention in all relevant 
contexts, are roughly comparable. One 
could argue that atypical agents work 
with individuals displaying more difficult 
problem behavior, thus providing an 
explanation for their lower success rates. 
However, as there is no consensus in the 
field as to what metric should be used to 
gauge level of difficulty, one cannot know 
whether this interpretation is correct. In 
addition, it is important to note that when 
typical agents implemented intervention, 
O 



they did so only after they had referred 
the case to one or more experts/profes- 
sionals who then helped them structure 
the intervention. In other words, the 
atypical agents were still involved albeit 
in a consultative role rather than as 
implementors. 

It is encouraging that typical 
intervention agents were able to produce 
good success rates, as ultimately these are 
the people who must carry the brunt of 
intervention in day-to-day situations. It is 
even more encouraging that the generally 
less controlled, less restrictive situations 
represented by typical settings produced 
success rates comparable to those 
obtained in atypical settings. Finally, it is 
most encouraging that intervention in all 
relevant contexts, truly a best practice, 
produced success rates at least as good as 
those obtained with more circumscribed 
intervention that did not address all 
relevant contexts. In sum, the ecological 
data suggest that the involvement of 
typical intervention agents may increase 
success rates and that implementing 
intervention in less restrictive circum- 
stances (i.e., typical settings, all relevant 
contexts) does not typically decrease the 
success rate much below the 50% level 
obtained generally. Apparently, with PBS 
intervention, it is possible to have both 
successful outcomes and ecological 
validity. 

Implications for 
Future Research 

We have now reviewed the database for 
studies that met our methodological 
criteria for inclusion. Earlier we noted the 
bias in this database toward articles that 
emphasized internal validity concerns. 
Although some portions of the database 
did address external validity concerns, it 



Discussion 



was clear that the bulk of the literature 
emphasizing these concerns is found in 
the excluded research articles and, to a 
great extent, among clinical reports and 
descriptive accounts provided by 
practitioners, parents, trainers, and others 
concerned with day-to-day support in 
naturalistic contexts. In Chapter 1, we 
alluded to the gap between the needs and 
interests of researchers and those of 
nonresearchers (consumers). Having 
reviewed the database, we are now in a 
position to weigh its strengths and 
weaknesses in light of nonresearcher 
concerns so we can formulate an agenda 
that addresses knowledge gaps by 
building on what is known while frankly 
acknowledging what is not known. This 
issue, of course, is the basis for the fifth 
and final research question to which we 
now turn our attention. 

How Responsive Is the PBS 
Literature to the Needs of 
Consumers (Nonresearchers)? 

Four priorities stand out in the literature 
as pertaining especially to the needs of 
consumers: (a) comprehensive lifestyle 
support, (b) long-term change, 

(c) practicality and relevance, and 

(d) consumer support issues. We will 
examine each of these in turn. 

Comprehensive Lifestyle 
Support 

Families and friends of people with 
developmental disabilities are focused on 
issues pertaining to family life, jobs, 
community inclusion, supported living, 
and expanding social relationships 
(Risley, 1996; Ruef, 1997; Turnbull & Ruef, 
1996, 1997; Turnbull & Turnbull, 1996), in 
short, comprehensive lifestyle support. 
The database, as a whole, does not reflect 
O 




this priority. First, only a tiny minority of 
outcomes involved goals of lifestyle 
change, implementation of lifestyle 
intervention, or reports of successful 
lifestyle change (Figure 15). Second, few 
data were reported on stimulus and 
response generalization (Table 8), 
measures that are indicative of broad 
change. Third, the preponderance of 
atypical settings (Figure 8), and the 
relative lack of intervention in all relevant 
contexts (Figure 14) are not responsive to 
family perspectives that stress the need 
for good adjustment to real-life settings 
and round-the-clock support to achieve 
this goal. Fourth, the paucity of interven- 
tions that combine stimulus-based and 
reinforcement-based interventions into a 
comprehensive multicomponent ap- 
proach (Table 6) also indicates that the 
emphasis is not on providing interven- 
tions that demonstrate efficacy across 
complex, changing contexts such as those 
that typify community settings. Fifth, the 
call for extensive environmental reorgani- 
zation as a crucial strategy for ensuring 
improvement in lifestyle (Risley, 1996) 
has not been heeded. Few outcomes were 
associated with this type of systems 
change (Table 12). 

Yet the gaps just noted need not 
remain a permanent feature of the 
research literature. Even in the current 
database, there were indications that this 
situation can improve. However, although 
they were in the minority, there were 
demonstrations of successful lifestyle 
change as well as stimulus and response 
generalization. Also, some investigators 
have, with good effect, intervened in all 
relevant contexts, used combined 
(multicomponent) interventions, and 
reorganized environments. There is also a 
clear trend toward implementing PBS 
more often in typical settings. Taken as a 
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whole, these data show that it is possible 
to respond to consumer needs for 
comprehensive lifestyle change. Indeed, 
this embryonic research base, when 
combined with similar information from 
the excluded research studies as well as 
numerous reports in the nonresearch 
literature, fully justifies a major shift in 
research priorities toward analysis and 
evaluation of comprehensive lifestyle 
support. As noted, all the elements of this 
approach are present in the literature. 
Generally lacking at the moment are (a) 
rules for their systematic combination 
and (b) the scientific scrutiny of those 
rules that is necessary to prove efficacy. 

Long-Term Change 

Consumers tend to be concerned about 
problem behavior over long periods of 
time. Families note that advocacy efforts 
necessary to achieve comprehensive 
lifestyle change can often take years 
(Nickels, 1996; Turnbull & Turnbull, 
1996). Both teachers and families state 
that transitioning individuals from 
preschool to elementary and high school 
and then to the workplace and supported 
living requires a lifespan perspective that 
views the successful management of 
problem behavior as a never-ending 
process responsive to different challenges 
at different stages of life (Turnbull, 1988; 
Vandercook, York, & Forest, 1989). Not 
surprisingly, then, the overall short-term, 
uncoordinated nature of programs and 
supports has all parents, even those who 
are currently satisfied with their pro- 
grams, worried about the future (Ruef, 
1997; Turnbull & Ruef, 1997). The 
database reveals a substantial gap 
between the needs of consumers for long- 
term demonstrations of efficacy and the 
interests of researchers who follow 
individuals for short periods of time, 
qT" typically for less than 6 months and 



in no case for more than 2 years (Table 9). 

There are, however, both empirical 
and conceptual grounds for believing that 
the gap can be closed. First, the database 
includes a limited number of demonstra- 
tions of successful maintenance effects 
lasting up to 2 years. There is no a priori 
reason for assuming that the effects 
cannot be further extended, especially 
given the excluded research studies and 
clinical reports, some of which note long- 
term maintenance. At a conceptual level, 
one might expect long-term effects if the 
PBS approach were implemented in a 
manner more consistent with its general 
philosophy. To the extent that deficient 
environments and deficient skills 
continue to be identified over time, which 
is almost always the case when one 
follows an individual over many years in 
changing life circumstances, PBS 
strategies would have to be added and/or 
modified. In other words, intervention 
never stops. This view is in contrast to 
many traditional studies in which 
maintenance is defined as durable 
success following intervention cessation 
(Carr et al., 1990). In a truly comprehen- 
sive PBS approach, maintenance would be 
guaranteed because intervention would 
never stop. Interestingly, an added benefit 
of such a long-term strategy is that, over 
time, the individual is supported in many 
different situations (a feature that would 
enhance stimulus generalization) and is 
taught many different skills (a feature that 
would enhance response generalization). 
Thus, comprehensive changes are likely to 
be facilitated over the protracted periods 
of time that PBS is in effect. In short, 
maintenance and comprehensive lifestyle 
change are intertwined variables. 

Practicality and Relevance 

Unless consumers view a research finding 
as being usable in their particular 
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circumstances, it is unlikely that it will 
guide day-to-day efforts to effect mean- 
ingful, long-term change. The gap 
between research and practice is high- 
lighted by the fact that researchers 
generally focus on methodological rigor, 
whereas consumers generally focus on 
practicality and relevance (Carnine, 

1997). 

Both interviews (Turnbull & Ruef, 
1996) and focus groups (Ruef, 1997) 
conducted with families indicate that 
relevance is most often defined as 
pertaining to lifestyle change. However, as 
noted before, this topic is dealt with only 
marginally in the database. Families are 
also concerned with a number of 
practical issues (Ruef, 1997) that go 
beyond the simple desire for information 
about the nature of specific interventions: 
(a) How many people are necessary to 
implement PBS, and can a parent do it 
alone? (b) How much training does a 
parent need, and must one become an 
expert to be successful? (c) What is the 
average number of hours per week 
necessary to effectively implement PBS in 
the first 6 months? After 6 months? After 
1 year? (d) Can one be successful by 
implementing only part of the PBS 
approach? (e) Are there shortcuts (e.g., 
simpler versions of functional assess- 
ment), and how does one sustain the 
energy levels necessary for long-term 
efforts? (f) How can parents effectively 
implement PBS when their own fears, 
embarrassment, and anger get in the way 
(maintaining composure)? (g) How can 
parents do an effective job in the face of 
negative reactions from friends, acquain- 
tances, family, and others in the commu- 
nity? The fact that the database rarely 
confronts these issues demonstrates the 
existence of a wide research-to-practice 
gap that argues in favor of a future research 
O t centering on practicality criteria. 
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With respect to other consumer 
groups, the issues may vary, but the focus 
on practicality and relevance does not. 
Teachers, for example, regard proper 
training for managing problem behavior 
as a top priority (Pearman, Huang, & 
Mellblom, 1997). Again, their concern is 
not just with the specifics of intervention 
but, rather, with how well intervention 
practices will fit into the system in which 
they work (Ruef, 1997): (a) How can PBS 
be implemented in a general-education 
classroom? (b) How does PBS vary as a 
function of grade level (elementary, 
middle school, high school)? (c) What 
structural modifications are required for 
implementation in special-education 
versus general-education settings? 

(d) What systems prerequisites must be 
satisfied for PBS to be a viable option 
(e.g., does it matter if you have one child 
with a disability in a class of typical 
children or a class composed entirely of 
children with disabilities)? (e) Does 
implementing PBS with one student take 
away time spent with other students 
(fairness)? The database rarely touches 
these topics, a fact that is particularly 
evident from the small number of 
outcomes associated with broad environ- 
mental reorganization, a key facet of 
systems change (Table 12). That teachers 
widely perceive research to be irrelevant 
to their needs is also evident from data 
showing that fewer than 1% of the 
nation’s 4 million teachers participate in 
the American Federation of Teachers 
Education Research and Development 
program in which emphasis is placed on 
using research to make informed 
educational decisions to guide practice 
(Billups, 1997). 

Other groups of consumers such as 
friends of people with disabilities, policy 
makers, and people with disabilities 
themselves raise questions about the 
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complexity of systems change and the 
relevance of current research for dealing 
with this complexity (Morrissey, 1997; 
Ruef, 1997). 

Several developments in the field 
point to a way of closing the research-to- 
practice gap to enhance practicality and 
relevance. Specifically the Participatory 
Action Research (PAR) approach sees 
researchers and consumers as collabora- 
tors (Meyer & Evans, 1993; Turnbull, 
Friesen, & Ramirez, 1998). Traditionally, 
researchers have defined the issues, 
formulated a plan for investigating the 
issues, and occasionally disseminated the 
results to consumers (but more often to 
other researchers). The PAR model, in 
contrast, views consumers as having an 
active rather than passive role. Consum- 
ers can play many roles that include 
helping to define the issues, assisting in 
the design of those aspects of the research 
that enhance ecological and social 
validity, and providing consultation on 
how to package the results so that they 
are more readily usable by other consum- 
ers. Policy makers have called for this 
type of collaboration for several years 
now (Lloyd, Weintraub, & Safer, 1997; 
Malouf & Schiller, 1995). However, with 
virtually no exceptions, the PAR model 
was not a feature associated with our 
database. Nonetheless, it is encouraging 
to note that in a recent issue of the 
Journal of Special Education devoted to 
research in severe disabilities, multiple 
researchers independently endorsed the 
idea of making the PAR model a critical 
component of a future research agenda 
for the field (Browder, 1997; Nietupski, 
Hamre-Nietupski, Curtin, & Shrikanth, 
1997; Reichle, 1997). In sum, although 
parents and teachers, for example, will 
typically not have the expertise to 
enhance the technical design aspects of 







research, they can and should play a 
major role in enhancing ecological and 
social validity and in identifying and 
resolving barriers to achieving successful 
outcomes. 

Although the database on social 
validity was scant (Figure 16), it is 
somewhat encouraging to note that the 
data were generally positive with respect 
to issues of feasibility, desirability, 
acceptability, and lifestyle change, 
suggesting that, at least for the limited 
sample of consumers involved, PBS was 
viewed as practical and relevant. The field 
needs to build on this base by developing 
research models that reflect the major 
elements of the PAR approach. In this 
regard, recent work on goodness-of-fit 
models (Albin, Lucyshyn, Horner, & 
Flannery, 1996) seem especially relevant. 
This research strategy stresses the notion 
that interventions must be congruent 
with contextual variables involving 
participant characteristics, characteristics 
of the people who will be implementing 
the plan, and systems features related to 
the environment in which the plan is to 
be implemented. Consumer needs and 
priorities are critical. Recent research 
demonstrates clearly that a focus on 
goodness-of-fit not only produces socially 
valued intervention outcomes but is the 
logical translation of the PAR philosophy 
into scientific methodology (Lucyshyn et 
al., 1995, 1997). 

Consumers Want Support, Too 

It is easy to misinterpret PBS as referring 
only to the person with disabilities, 
namely, how best to support that person 
through skills training, environmental 
reorganization, and a focus on improving 
lifestyle. However, as we noted, PBS refers 
to systems change broadly conceived, that 
is, all elements of the system including 
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the needs of other people who must 
support the person with disabilities. Thus, 
a critical theme that has emerged in the 
literature is that consumers such as 
parents and teachers require support. 

Some researchers have argued 
forcefully that professional support 
should be family-centered and entail 
comprehensive systems change involving 
all relevant parties and not just the 
person with disabilities (Albin et al., 

1996; Dunlap & Robbins, 1991). Nonethe- 
less, parents continue to report a lack of 
family-centered service delivery systems 
and an overall lack of support from 
professionals paid to deliver services 
(Wheeler, 1996). Few examples exist in 
the literature that demonstrate support 
for the people (families) who must carry 
the weight of supportingothers (c.f., 
Lucyshyn et al., 1995, 1997; The Family 
Connection staff et al., 1996; Turnbull & 
Turnbull, 1996). 

Interestingly, although both parents 
and teachers value advice and training 
from professionals, they particularly 
value parent-to-parent and teacher-to- 
teacher mentoring (Gersten & 
Brengelman, 1996; Gersten, Morvant, & 
Brengelman, 1995; Ruef, 1997; Santelli, 
Turnbull, Marquis, & Lerner, 1993, 1995). 
Teachers also value emotional support 
from principals and other administrators 
as a way of coping with high job stress in 
working with challenging populations 



(Fimian, 1986; Littrell, Billingsley, & 

Cross, 1994). 

In some limited respects, the 
database touches on several of the 
concerns just described. Thus, the fact 
that typical intervention agents working 
in typical settings are sometimes 
intervening in all relevant contexts (Table 
13) is a step in the right direction, 
because this pattern implies family- 
centered and school-centered services. 
However, as noted previously, this pattern 
is associated with only a minority of 
outcomes in the database. More signifi- 
cant perhaps, the database does not show, 
in any systematic way, a concern for the 
broader needs of consumers that involves 
issues such as peer mentoring, emotional 
support, and stress reduction. Instead, a 
premium is placed on teaching specific 
intervention strategies at the expense of a 
focus on the broader systems issues 
related to supporting consumers who will 
be responsible for facilitating lifestyle 
changes for decades (i.e., parents often do 
this for 50 or more years, and siblings can 
have this role for even longer). This gap in 
the research literature can be addressed 
only by enlarging the conception of best 
practices to include the systematic assess- 
ment of consumer needs and by acting on 
those needs by designing supports for the 
supporters — families, teachers, job coaches, 
and other community-based staff. 
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CHAPTER 5 

SUMMARY 



We now summarize the findings with the 
greatest clinical and conceptual signifi- 
cance as they relate to the five research 
questions posed at the beginning of this 
review. 

How Widely 
Applicable is PBS? 

1. The approach is widely applicable to 
people with serious disabilities who 
exhibit serious problem behavior. 

2. The approach can be applied by 
typical intervention agents in typical 
settings. Its use is not restricted to 
experts operating in specialized 
circumstances although a majority of 
outcomes are still associated with this 
pattern. 

In What Ways is 
the Field Evolving? 

1 . PBS is not a fad. It has been showing 
steady and dramatic growth especially 
over the most recent time period 
reviewed. 

2. Outcomes involving the most difficult 
problem behavior (e.g., SIB and 
combinations of problem behaviors) 
have become more numerous. 

3. Generally, typical and atypical 
intervention agents are about equally 
likely to conduct PBS intervention. 
With respect to intervention settings, 
atypical settings are more likely to be 
the venue for intervention. However, 
there is a steady increase over time in 
the use of typical settings, and the gap 
between the use of typical versus 
atypical settings is closing. 

O 




4. There has been a dramatic increase 
over time in the use of assessment 
prior to planning intervention. 

5. Both formal functional analysis and 
combination assessments have 
become more numerous. There is 
concern, however, as to whether, 
functional analysis is a practical 
method in naturalistic contexts. 

6. There is a clear focus on escape- 
motivated problem behavior and on 
problem behaviors that are multiply 
motivated. 

7. Repeated assessments have remained 
uncommon. When they do occur, they 
are likely to be carried out by typical 
intervention agents and in typical 
settings. 

8. Stimulus-based intervention has 
gradually become more common than 
reinforcement-based intervention. 

9. Combination interventions do not 
show an increasing trend. When they 
do occur, they are likely to be carried 
out by typical agents and in typical 
settings. 

10. With respect to systems change, 
significant others are increasingly 
likely to alter their behavior as part of 
the intervention. No trend is seen, 
however, for environmental reorgani- 
zation, a critical aspect of systems 
change. 

11. Intervention in all relevant contexts 
does not show a trend. When it does 
occur, it is likely to be done by typical 
agents and in typical settings. 

12. There is an increase over time in 
outcomes associated with short-term 
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maintenance (5 months or less) but 
none for longer-term maintenance. 

13. Outcomes associated with generaliza- 
tion, lifestyle change, and social 
validity show no trends and are few in 
number. * - 

How Effective is PBS? 

1 . Modest to substantial increases in 
positive behavior are typically 
observed following the application of 
PBS intervention. 

2. With respect to reduction in problem 
behavior, about one-half (using a 
criterion of 90% or more decrease in 
problem behavior from baseline) to 
two-thirds (using an 80% criterion) of 
the outcomes are successes. 

3. Typically, the success rate does not 
change as a function of whether 
stimulus-based intervention and 
reinforcement-based intervention are 
used alone or in combination with 
each other, nor does it change when 
non-PBS interventions are added. 

4. With respect to maintenance effects, 
about two-thirds of the outcomes are 
successes (using a 90% reduction 
criterion). However, the database is 
small and gets smaller as the duration 
of follow-up is lengthened. 

5. There are demonstrations of success- 
ful lifestyle change and good social 
validity, but these data are reported 
only for a very small minority of 
outcomes. 



What Factors Modulate 
the Effectiveness of PBS? 

1. Intervention for combinations of 
problem behavior produces lower 
success rates than interventions for 
single types of problem behavior. 

2. The success rate (using a 90% 
reduction criterion) for interventions 
based on a prior functional assess- 
ment was almost twice that obtained 
when this type of assessment was not 
conducted. 

3. The success rate (90% criterion) was 
greater following interventions that 
involved systems change (although 
the database for one aspect of systems 
change, environmental reorganization, 
was very small). 

4. The success rate (90% criterion) 
associated with typical agents was 
higher than that obtained with 
atypical agents. The success rates in 
typical versus atypical settings were 
comparable. 

How Responsive is 
the PBS Literature to the 
Needs of Consumers 
(Nonresearchers)? 

1 . Comprehensive lifestyle support is a 
major goal of families, but the 
database rarely addressed this issue. 

2. Families are most concerned with 
long-term behavior change. The 
database, in a minority of cases, 
demonstrated such change. However, 
no outcomes involved follow-up of 
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longer than 2 years, which poses 
difficulties for the vast majority of 
families that think of maintenance in 
terms of decades, rather than months. 

3. Consumers (e.g., parents, teachers) 
judge interventions in terms of their 
practicality and relevance and are 
concerned with how well intervention 
plans mesh with the realities of the 
complex social systems in which the 
consumers must function. The 



database, more concerned with issues 
of rigor and demonstrations of 
experimental control, generally failed 
to focus on larger consumer goals. 

4. Consumers are concerned with 
obtaining support for themselves in 
addition to support for people with 
disabilities. This topic was not a focus 
of systematic research in the studies 
included in the database. 
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For Researchers 

The main recommendation to researchers 
is that we need a new applied science. For 
a truly applied science to develop, 
researchers need to address consumer 
needs more systematically and more 
frequently. A robust applied science will 
require changes in assessment and 
intervention practices, a redefinition of 
what constitutes a significant outcome, 
and measurement procedures that 
address these new priorities. 

With respect to assessment, methods 
must be developed that are user-friendly, 
are feasible in the community, and yield 
accurate information. Functional analysis 
meets the last criterion but not the first 
two; it therefore needs to be supple- 
mented and often replaced with a new 
generation of assessment tools that meet 
all three criteria and can be implemented 
repeatedly as circumstances warrant. 

With respect to advancing interven- 
tion practices, researchers need to focus 
on consumer goals pertaining to compre- 
hensive lifestyle support, long-term 
change, and direct support to consumers 
themselves. These goals can most 
plausibly be met by an increasing 
emphasis on multicomponent interven- 
tions that are linked to assessment 
information, broad reorganization of 
context (systems change), an emphasis on 
ecological validity (typical agents and 
typical settings), intervention in all 
relevant contexts, and the application of 
all these practices over protracted periods 
of time. The consumer goals just noted 
together with the goals of practicality and 
rpWance can be best met by focusing on 



how well intervention practices fit with 
specific contexts (goodness-of-fit), and 
this determination can be made through 
the active participation of consumers in 
the research process as advisers and/or 
collaborators (Participatory Action 
Research). 

Finally, with respect to outcomes, the 
PAR model needs to be adapted as well in 
order to define, in specific terms, the 
dimensions of new outcome measures 
defined primarily in terms of long-term, 
socially validated, comprehensive lifestyle 
change and only secondarily in terms of 
reductions in problem behavior. 

For Service Providers 

The main recommendation to service 
providers is to fix problem contexts, not 
problem behavior. Problem contexts (i.e., 
environmental deficiencies and skill 
deficits) are the fertile ground from which 
problem behavior springs. Therefore, 
whenever problem behavior is identified 
as a clinical issue, service providers ought 
first and foremost to structure interven- 
tion so that it reflects a knowledge 
(derived from systematic assessment) of 
what is wrong with the individual’s 
environment (e.g., educational practices, 
scheduling issues, lack of control) and 
skill repertoire (e.g., lack of communica- 
tion, poorly developed social skills, 
insufficient self-management). Such 
assessment information can then be used 
to redesign the environment and enhance 
the individual’s adaptive skill repertoire. 
The primary effect of this approach is to 
strengthen positive behaviors. An 
important but secondary effect is to 
produce a decrease in problem behavior. 
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Reflecting consumer needs, service 
providers need to offer hands-on, ongoing 
(rather than episodic) support to families, 
teachers, and staff over long periods of time. 
This strategy has the dual effect of support- 
ing the consumers themselves and 
gradually permitting a transfer of support 
tactics from the professional service 
provider to typical intervention agents. 

The PAR model for researchers is also 
appropriate for service providers. Specifi- 
cally, service providers need to transform 
their role from experts who unilaterally 
select goals and structure interventions,^ 
collaborators who, in consort with consum- 
ers, define the dimensions of comprehensive 
lifestyle change (thereby ensuring social 
validity) in a manner that represents a good 
fit with the reality of day-to-day contextual 
constraints (thereby ensuring practicality 
and relevance). 



For Social 
Policy Advocates 

The main recommendation for social 
policy advocates is that regulations 
defining quality of services need to 
mandate standards of best practice. These 
standards should involve or address: 



• repeated functional assessments that 
identify, on an ongoing basis, the 
environmental and behavioral 
deficiencies that are the root cause of 
problem behavior; 

• direct linkage between assessment infor- 
mation and the design of interventions; 



• intervention in all relevant contexts, a 
strategy that almost invariably means 
the use of multicomponent interven- 
tions geared toward altering systems, 
not just discrete behaviors; 

• ecologically valid relevant contexts 
(i.e., typical agents carry out interven- 

O on in typical settings); 
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• the long-term perspectives of 
consumers — by designing and 
redesigning interventions as changes 
in life circumstances warrant; that is, 
intervention plans must have a 
lifespan orientation rather than a 
crisis management orientation; 

• consumers being an integral part of 
the system by constructing interven- 
tions that respond to the personal 
needs and concerns of consumers 
(goodness-of-fit) thereby ensuring 
practicality and relevance; 

• social validity issues, defining 
outcome goals in terms of compre- 
hensive lifestyle change and support 
and not just reduction in problem 
behavior. 

For the Government 

The main recommendation for govern- 
ment is that resources should be invested 
to ensure the continued development and 
evaluation of a truly applied science of 
PBS that is sensitive to consumer needs. 
Government can strengthen this process 
by (a) developing grant competitions that 
require adherence to best practices, 
emphasizing the demonstration of 
socially valid comprehensive lifestyle 
change rather than microanalysis of 
cause-and-effect processes in situations 
that lack ecological validity; (b) creating a 
nationally accessible database on PBS 
that is updated periodically so that 
consumers, advocates, policy analysts, 
and researchers can study the database to 
see what progress is being made, what the 
gaps are, and what future directions 
might be fruitful; (c) convening periodic 
state-of-the-art conferences that define 
where we are and how PBS needs to 
change to keep abreast of new develop- 
ments in policy, advocacy, consumer 
needs, and research findings. 
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